Its 2008 and my project goes on
It was May 2007 that I made my last post. That's a long time. The thing that made me remember this blog is I am working on filesystems and just had an idea. Currently in my project I am in the verge of finishing off with paging and memory management. I had some good progress in the past few months, but I still am thinking of what to do about the filesystem. I have a good idea of how to write the VFS. I didn't write it yet because I want to make my mind first, as to what I am going to do about file handling in general. Filesystems are very complex. There's no way I could do the OS and get my hands dirty with another filesystem. An modern fs like reiserfs is something like 50k - 80k lines of code?
Some good ideas though that come to my mind:
1) In plan 9, files are used for stream management. Plan 9 streams (which had a modular design but wasn't very successful) are used for device i/o, and networking.
ioctl calls and sockets are replaced by 2 files with suffix 'ctl' and 'data'. Now in ReiserFS4, it is suggested that if a lot of small files can be efficiently stored on the block device, one could implement file attributes as files, which would be accessed by a special path just as '..' and '.', such as '....' but using the conventional read/write calls. This is not a bad idea, and can also make it possible to implement Plan 9 'ctl' and 'data' files as file attributes for that device (e.g. an ethernet device, or a tcp connection).
2) Memory is a stream of bytes, and a block device is a stream of bytes. Logically they are the same. Now to manage access to memory efficiently, we use virtual memory. The physical memory pages are managed in the background for us, in the most efficient way possible by maximising sharing, but yet we partition a block device and access it by its real address. Furthermore this dividing of block devices are so opaque that they are even escalated to the user of the system by tools like fdisk. What I think would be really useful is that the filesystem itself hides from the user the partitioning or managing of a set of disks and present the user with just the simplicity of raw space. In that respect the idea of ZFS sounds really good. One downside however, is that its IO scheduling etc. is based on disks, since its more of an enteerprise filesystem. A more interesting design would be to handle both solid state flash disks and disk drives in a transparent way.
Anyways what I am thinking is that the (1) needs changes in existing VFS desings, since I haven't even written it, I might leave support for a special attribute accessor path component for future experimentation. The other idea I have is that I might maybe design a filesystem that has no journalling protection or good performance, but something that is dead simple and flexible. Something that compresses data for reducing I/O, that duplicates data for simple recovery and that allows flexibility of partitions. Maybe I might get away with creating something simple and useful instead of putting all that effort to port an existing huge filesystem. Plus I would have learned something ;-)
Some good ideas though that come to my mind:
1) In plan 9, files are used for stream management. Plan 9 streams (which had a modular design but wasn't very successful) are used for device i/o, and networking.
ioctl calls and sockets are replaced by 2 files with suffix 'ctl' and 'data'. Now in ReiserFS4, it is suggested that if a lot of small files can be efficiently stored on the block device, one could implement file attributes as files, which would be accessed by a special path just as '..' and '.', such as '....' but using the conventional read/write calls. This is not a bad idea, and can also make it possible to implement Plan 9 'ctl' and 'data' files as file attributes for that device (e.g. an ethernet device, or a tcp connection).
2) Memory is a stream of bytes, and a block device is a stream of bytes. Logically they are the same. Now to manage access to memory efficiently, we use virtual memory. The physical memory pages are managed in the background for us, in the most efficient way possible by maximising sharing, but yet we partition a block device and access it by its real address. Furthermore this dividing of block devices are so opaque that they are even escalated to the user of the system by tools like fdisk. What I think would be really useful is that the filesystem itself hides from the user the partitioning or managing of a set of disks and present the user with just the simplicity of raw space. In that respect the idea of ZFS sounds really good. One downside however, is that its IO scheduling etc. is based on disks, since its more of an enteerprise filesystem. A more interesting design would be to handle both solid state flash disks and disk drives in a transparent way.
Anyways what I am thinking is that the (1) needs changes in existing VFS desings, since I haven't even written it, I might leave support for a special attribute accessor path component for future experimentation. The other idea I have is that I might maybe design a filesystem that has no journalling protection or good performance, but something that is dead simple and flexible. Something that compresses data for reducing I/O, that duplicates data for simple recovery and that allows flexibility of partitions. Maybe I might get away with creating something simple and useful instead of putting all that effort to port an existing huge filesystem. Plus I would have learned something ;-)
posted by Bahadir at 11:11 pm
0 Comments:
Post a Comment
<< Home