Comment by ambrop7

10 years ago

Related, I'm doing a FAT32 driver for embedded systems (fully asynchronous!). It's part of my APrinter firmware project: https://github.com/ambrop72/aprinter/tree/master/aprinter/fs

Currently it has good read support and limited write support (can re-write existing files but not append or create new files).

Before writing that code I made some prototype read-only code in Python, to make sure I understand the FS structure properly: https://github.com/ambrop72/aprinter/blob/master/prototyping...

5 comments

ambrop7

Reply
userbinator  10 years ago

Embedded as in "32-bit ARM running Linux", or "8-bit microcontroller"? Either way, as someone who has written a FAT driver for embedded systems in the later category, that looks like it's far more code than it needs to be. Full read/write functionality can be done in approximately 800 (Z80) machine instructions. FAT is a linked list, and if you look at it that way, it doesn't take much code to manipulate one.

I've written a bit more about that before:

https://news.ycombinator.com/item?id=7492318

Append = find an empty cluster and link it into the chain for the file, then write into it. Create is similar except you start with a empty chain and also add an entry to the directory (which is like writing/appending to a file, because directories are files.)

When allocating next clusters you can reduce fragmentation significantly over the dumb "first fit" strategy if you scan the FAT to find contiguous free clusters and apply a next/best/worst fit. Even better if you add an API to allow tuning the amount of "gap" after a file when creating it, based on knowledge of how much it may expand in the future.

I believe that with good tuning and allocation heuristics, FAT32 can outperform the far more complex filesystems (ext*, NTFS, etc.) widely believed to be superior. One idea I've never gotten around to testing out is to modify the Linux FAT driver and do some benchmarking.

  • JohnBooty  10 years ago 

       I believe that with good tuning and allocation 
   heuristics, FAT32 can outperform the far more complex
   filesystems (ext*, NTFS, etc.) widely believed to be
   superior. One idea I've never gotten around to testing
   out is to modify the Linux FAT driver and do some
   benchmarking.

    I don't understand. Has anybody ever claimed that the more complex alternatives were actually faster?

    NTFS and other filesystems that followed FAT32 are "superior" because they support things like journaling and more robust permissions... things that unavoidably incur (a least) a small performance hit.

  • ambrop7  10 years ago

    It is targeted to ARM uC's and also works on AVRs with enough RAM and program space (yes I've tested it). Write support requires a significant amount of RAM; I think about six 512-byte blocks. When compiled in read-only mode, it should need no more than two.

    One reason for the RAM needs is that I use a block cache and I take care do to "atomically" certain related operations (on the block cache level). These are things like marking a cluster as allocated, decrementing the number of free clusters in FsInfo, and linking the allocated cluster to the end of a chain. These things require multiple blocks to all be present in the block cache at the same time.

    I suppose the asynchronous design also contributes to the code and RAM size (though I did not actually measure the code size or compare it to other drivers). It may look like more that it really compiles to. Anyway would be very surprised if you show me another asynchronous driver - I couldn't find any!

    Also consider that my code supports VFATs (decoding only, file creation is not supported anyway).

    The design is intentional, I have sacrificed some RAM/program size for asynchronous operation and better reliability in case of random write failures (because of the block cache, all writes can reliably be retried later, avoiding corruption as long as they eventually complete).

    Update: I estimate the code compiles to about 10kB with gcc for Coretex-M3 using -Os. This is certainly within limits of most ARM uCs and many AVRs.

voltagex_  10 years ago

You got further than me. I got stuck trying to find the files themselves in a disk image. I think the code is in my junkcode repo on GitHub. My end goal is ext4, and then on to a FAT32 over USB gadget hack to bring back mass storage support to Android.

What did you use as your main source of documentation while developing?