lazy weekend

I was meant to do a lot of work this weekend, namely building tcbs, initial tasks, loading and switching to them. Instead I ended up playing games, watching movies and eating a lot of chocolate ice cream. Next weekend I'm in london for the wedding celebration of a friend.

Anyway it sounds very lame, but the one thing I actually did do was thinking while playing. It's not really thinking, but when you come across a problem you don't know how to solve yet, it really helps to do absolutely irrelevant things for a few hours. It shouldn't be something that requires too much concentration, but it should still have you involved so that you get rid of stress, like cooking, or playing games. While you're doing something, under the hood you give yourself time to think of a solution, and suddenly something pops up in your head. And that thing is usually a shortcut that would greatly ease your work compared to the useless progress you would have done if you actually continued to work straight after you stumbled upon the problem. What's even worse, is you've spent that time working rather then clearing up your mind. So the conclusion is, sometimes playing games or reading the papers (even for a few hours) is legitimately acceptable and part of engineering. I've also seen this on an article on some IBM website written by a hacker.

some more progress

OK. So far, I wrote 3 types of memory allocators, each coming to about ~400 lines of code. The first is the page allocator, second is a kmalloc on top of that. The third is the concept of an extremely basic memcache. Page allocator internal state depends on the memcache, and kmalloc depends on the page allocator. I've also spent considerable time for incorporating a test architecture, that builds a test executable linking with all these allocators. This accepts commandline arguments, and a python script runs random allocation tests on all three allocators using different page_size, allocation size, number of allocations and other similar parameters. It turns out this sort of unit/regression testing is very powerful, as each time I make a change in either of them, I can run these tests and a bug usually quite soon shows up. Also another very useful thing was using BUG() and BUG_ON() macros, which pretty much right after I ran the allocators were caught, and immediately pointed out at "what" was going wrong. A lot of saving of wasteful bug searching hours. I spent so much time on incorporating this "test" architecture, and the time I spent is justified really. The only problem is that I am finding difficulty picking up clues of bugs to test and compare. If you dig deep into the internal state of each allocator, the test cases get very specific, and when you make a rewrite of the allocator, you need to almost rewrite the test as well, which is bad. But then if you just focus on what the interface functions return, something might have gone wrong very early and show up very late after you do, say 100 allocations.

Anyways, the allocators are so far doing well. The page allocator - I plan to use as the primary page allocation scheme on the whole system. The microkernel will allocate all physical memory through these, and then grant them to pagers, which in turn will grant them (or just map) to userspace applications. But the microkernel must be in full control of all resources first.

The memcache - It incorporates a bitmap allocator for fixed size structures. I used it so far primarily for allocating page middle directories (pmd). Very efficient, useful and simple. Allocation and deallocation is o(1) (ignoring the stepping through bitmap words) thanks to the bitmap allocation scheme. Fragmentation is also none, since all structures get fully utilised. The only downside is one has to preallocate a chunk of memory initially for the memcache.

Kmalloc - I haven't made use of it yet. I originally wrote it with tcb allocations in mind, but I may just do that with the memcache.


More stuff I've done:
I've debugged add_mapping()/remove_mapping() functions. It took hell of a lot of time to find out the cause of an ever-occuring dreaded data abort during the flushing of the caches. It was impossible what the cause of this was, because it was the debugger that caused it, while stepping through the cache flush code. I still couldn't figure out why, but I know that it's the debugger because if I dont step through it aborts don't occur. But because I am pessimistic and really want to get things right before moving on, I thought it's a bug in my code, and tried hard to find out why it was happening.

I also gladly found out yesterday that QEMU supports the versatile pb926 platform. I booted my code on it with no modification. Also got insight working with QEMU. It's really fantastic the two together. This essentially means I can continue development anywhere I want with just a pc. Also it's great to have a v5 mmu-capable arm cpu model at hand.

That's it for now. Future plans: To get tcbs sorted, also loading of a userspace-task, and switching between the microkernel and that task. This thing is getting more and more fun. Especially while listening baroque music from www.bach-radio.com. Bach and Vivaldi really rock.