µC/FS White Paper

You are here : Micrium : Products : µC/FS : White Paper

Overview Benefits PC Windows Evaluation Drivers Performance and resource usage Licensing Pricing Information

Embedded Flash based File Systems

With faster and larger processor architectures, developers and customers are expecting more from their embedded systems. Out of the list of peripherals and capabilities people are now expecting from an embedded system, the availability of a file system is becoming a no-brainer. We just have to think at the number of devices that offer flash storage.

A few design choices need to be made when deciding to use a file system in an embedded design. Amongst the decisions, here are some of them:

1. The type of file system structure required (Microsoft FAT,  Linux EFS or proprietary)
2. Files compatibility with another Operating System native file system.
3. The storage medium type to be used
4. The requirement for a removable medium.

All these items have an impact on the hardware/software design.
In an embedded system, the storage medium is often not a Hard Disk Drive. It is more often a Flash based medium. The main design constraint relates to the way the file system manages the data on the storage medium in case of power reset or failure.

On a system with a Hard Disk drive and a good power supply, the Operating System File System has means to guarantee that the file system can recuperate from a power reset or failure.
On an embedded system with a Flash based medium, the minimum data element (sector or iNode) need to be erased before it is rewritten. This requirement add constraints to the design that are different then those of a PC running Windows or Linux.

 In addition, if your medium is a removable medium, disconnecting the medium from the embedded system while the file system is making operations on it, can cause corruption. This is similar to the requirement to click on the “Safe to Remove” tray icon in Windows before removing a USB Mass Storage device.
Removing the medium or loosing power will produce similar effects.

Depending on the type of file systems used, there are ways to make an embedded system with a Flash based storage medium “Power fail safe”.

Embedded device users make purchase decisions based on ease of use, speed of operation (they don't want to wait for images to load, or contacts to be accessed), and reliability, among other factors. Reliability and performance are therefore the critical elements of the flash file system. A good example of such devices is a cell phone or a PDA.

Typical File System structure

API layer
The API layer is the interface between the File System and the user application. It contains file functions such as FS_FOpen(), FS_FWrite() etc. The API layer transfers these calls to the file system layer.

File system layer
The file system layer translates file operations to logical block (sector) operations.
After such a translation, the file system calls the logical block layer and specifies the corresponding device driver for a device.

Logical block layer
The main purpose of the logical block layer is to synchronize accesses to a device driver and to provide an easy interface for the file system layer. The logical block layer calls a device driver to perform a block operation. This is also where the cache mechanism is located.

Device driver
Device drivers are low-level routines that are used to access sectors of the device and to check status. It is hardware independent but depends on the storage medium.

Hardware layer
These are the low level routines to access your hardware. These routines simply read and store fixed length sectors. The structure of the device driver is simple in order to allow easy integration of different hardware.

Flash translation layer
The flash translation layer establishes the link between "logical sectors" or "blocks" and "physical sectors".
A logical sector/block is the base unit of any file system. Its typical size is 512 bytes but can also differ.
A physical sector is an array of bytes on the flash chip that are erased together (typically between 2 Kbytes to 128 Kbytes). This is valid for every type of Flash device.

The flash translation layer is an abstraction layer between these two types of sectors.

Every time a logical sector is being updated, it is marked as invalid and the new content of this sector is written into another area of the flash. The physical address and the order of physical sectors can change with every write access. Hence, there is no direct relation between the sector number and its physical location.The flash translation layer manages the logical sector numbers by writing it into special headers. It does not matter to the upper layer were the logical sector is stored or how much flash memory is used as a buffer. All logical sectors (starting with Sector #0) always exist and are always available for user access.

File system types

The typical file system types are FAT, journaling and transactional.

• FAT is the traditional file system, originally developed for use in desktop systems or servers, using a file allocation table which tracks how files and directories are stored throughout the storage media. FAT file systems lack a means of synchronizing data in case of system failure and are therefore susceptible to corruption. We often refer to FAT as Microsoft FAT as it is the most used file system as it was used by DOS.
• Journaling file systems keep a log of all file system activities. Although more robust than a FAT file system, journaling file systems offer improved reliability and the expense of heavy system resource use. JFFS is an example of a Journaling Flash File System.
• Transactional file systems were developed specifically for embedded systems; they preserve both user and metadata through committing the known good state of the file system to memory with an atomic transaction point event.

Reliability

Atomic writes

To guarantee device data integrity, regardless of whether it is being used in optimum or adverse conditions, data must be either fully written to sectors on the storage media or not touched at all. The mechanism in the flash based file system that preserves data structures should be automatic and invisible to the developer. Additionally, it should ensure data integrity in the file system as well as the block device driver.

Wear-leveling and bad block management

Flash devices require strict management of the data blocks via software to ensure the chip expected lifetime is optimized and avoid write errors. A flash device must be erased before data can be changed. Wear-levelingalgorithms ensure that erase zones are used evenly across the entire flash device. Wear-leveling is especially useful when large portions of the flash device are used for long-term data storage. For example, a single configuration file may be written to an embedded device once and stored (and accessed) over an extended period of time.
In addition to wear-leveling, NAND flash requires a bad block management scheme that efficiently detects blocks that cannot be written to and reserves replacement blocks on the flash device. When properly implemented, bad block management will extend the life of a flash device.
Performance

As in many industries, the pressure by management and consumers to bring to market new embedded devices that perform complex tasks quickly, is a fact of like. The flash based file system is one of the sub-systems in a device that must contribute to this goal. Some of the features of the flash based file system to support this demand are:

Multi-threading

One of the characteristics of a flash device is to have faster read times than write times.. In systems that support multithreaded operations, read requests can be allowed to interrupt the more time-consuming write and erase operations. For example, a device with mobile TV applications that supports multi-threaded operations will improve the performance of streaming media applications.

Garbage collection (also referred a Background compaction)

As mentioned above, flash memory requires intelligent software to conduct write and erase operations. In order to keep valid data while throwing away unwanted data. An element of the flash device that was previously used and freed later on contains invalid data. Before this element can be reused, it needs to be erased. This process sometimes occurs during write operations, in order to properly complete a write. However, write speeds can be optimized when the garbage collection takes place during the flash based file system idle time, in preparation for a future write. This is referred to as background compaction, and results in a significant write performance improvement.

Boot times

The boot process of a flash file system involves mounting the flash drive followed by mounting the file system. The file system mount varies greatly depending upon the type of file system. This is the quick way to boot a system. If integrity is a mandatory requirement, mounting the flash disk requires a scan of the entire disk. The length of the scan process is directly proportional to the size of the disk. File systems such as FAT and Journaling use a lengthy CHKDSK utility to scan the file system integrity and perform repair operations if necessary. These file systems can take up to several minutes to mount. The fastest and most reliable option is a transactional file system, which completes by reading only two data blocks (the committed state and the known good state) and finishes in less than one second.

Other factors
As mentioned earlier, a key driver is delivering products to market faster than the competition, within an already constricting development schedule and budget. The market demands push design constraints beyond performance and reliability. The development process must take into consideration other technical subjects.
Amongst these subjects, ease of integration into the overall system design is a key consideration. Often, a system design involves a number of different vendors who must integrate their products to operate as one. Setbacks during integration are common, but costly. A software solution that is easy to integrate with other system components can reduce the risk of delivering a product to the market on time. Software that can be reused on future device designs with minimal modifications has greater benefits in reducing delivery risk. Multiple studies have demonstrated that using COTS products is the best way to minimize development cost and to reduce time-to-market.

Here are some of flash based file system features that may speed integration:

Hardware support

During production, it is likely that flash parts from multiple vendors will be used. The number of different parts (NOR or NAND) is considerable. A flash based file system solution that is limited to supporting one type of flash (i.e. NAND only) or parts from only one vendor is not a good choice for any high-volume device manufacturer.
The ideal software should auto-detect and support any flash technology (NAND and NOR as well as advanced flash technologies such as NAND controllers on processors, Multi-Level Cell and hybrid flash). Devices that use more than one type of flash (i.e. NOR to store application code, NAND to store data) will also benefit from a flash based file system that allows for writes to varying block sizes with a single instance of the software.

Portability

As we want our flash based file system to integrate with a variety of hardware, we also want the flash based file system to not be tied to a single operating system environment or application interface.
We want our flash based file systems to interface easily with the operating system, the flash hardware, the application code, and the development tool chain. These four factors provide for quick integration. Flash based file systems that can coexist with an operating system’s native file system offer further flexibility to designers that wish to take advantage of the selected operating system. Support for Unicode file names, long file names and support for POSIX are other factors which enhance the ease of portability of a flash based file system.

Source code and documentation provided

It is always a good practice to select a vendor that can provide you the source code of the product you are purchasing. With full source code, it is simple to use the flash based file system into a wide range of products.  The same statement can be made for documentation.
Scalability

Usually, when a manufacturer develops a product using a file system, other products follow shortly after. Even if source code availability and excellent documentation are important factors in a system developed for the first time, the selected software integration should provide a head-start for following designs.

In selecting a flash file system that can be used as a standard across multiple platforms, the solution must be scalable.
It would be preferable if the file system configuration could be done at run-time versus compile-time, allowing for various system settings and performance.  Flexibility across platforms includes support for multiple disk partitions and multiple file system volumes.

Vendor issues

Conclusion

With flash as the storage medium, data integrity mechanisms are crucial. Each physical block in a flash device has a maximum number of erase cycle. Wear-leveling and bad block management algorithms including power fail safe features are thus a minimum requirement for any flash driver.

To optimize the performance of the file system, the read and write operations should be enabled through multi-threaded support and background garbage collection to erase the used physical sectors in preparation for their next use.
A good file system should also allow for quick boot times. It is always annoying having to wait for a device to come up, whatever the device is!

Finally, the file system vendor should provide a wide range of supported hardware and operating systems. This is without mentioning the availability of quality source code and documentation. The last statement may look trivial, but it is not. Not all vendors have good quality standards.

Christian Legare
Vice-President, Micrium