docs: Add littlefs docs and a filesystem tutorial.
parent
f2650be844
commit
9a849cc7ca
|
@ -56,10 +56,14 @@ This class gives access to the partitions in the device's flash memory.
|
||||||
Returns a 6-tuple ``(type, subtype, addr, size, label, encrypted)``.
|
Returns a 6-tuple ``(type, subtype, addr, size, label, encrypted)``.
|
||||||
|
|
||||||
.. method:: Partition.readblocks(block_num, buf)
|
.. method:: Partition.readblocks(block_num, buf)
|
||||||
|
.. method:: Partition.readblocks(block_num, buf, offset)
|
||||||
.. method:: Partition.writeblocks(block_num, buf)
|
.. method:: Partition.writeblocks(block_num, buf)
|
||||||
|
.. method:: Partition.writeblocks(block_num, buf, offset)
|
||||||
.. method:: Partition.ioctl(cmd, arg)
|
.. method:: Partition.ioctl(cmd, arg)
|
||||||
|
|
||||||
These methods implement the block protocol defined by :class:`uos.AbstractBlockDev`.
|
These methods implement the simple and :ref:`extended
|
||||||
|
<block-device-interface>` block protocol defined by
|
||||||
|
:class:`uos.AbstractBlockDev`.
|
||||||
|
|
||||||
.. method:: Partition.set_boot()
|
.. method:: Partition.set_boot()
|
||||||
|
|
||||||
|
|
|
@ -178,6 +178,35 @@ represented by VFS classes.
|
||||||
|
|
||||||
Build a FAT filesystem on *block_dev*.
|
Build a FAT filesystem on *block_dev*.
|
||||||
|
|
||||||
|
.. class:: VfsLfs1(block_dev)
|
||||||
|
|
||||||
|
Create a filesystem object that uses the `littlefs v1 filesystem format`_.
|
||||||
|
Storage of the littlefs filesystem is provided by *block_dev*, which must
|
||||||
|
support the :ref:`extended interface <block-device-interface>`.
|
||||||
|
Objects created by this constructor can be mounted using :func:`mount`.
|
||||||
|
|
||||||
|
See :ref:`filesystem` for more information.
|
||||||
|
|
||||||
|
.. staticmethod:: mkfs(block_dev)
|
||||||
|
|
||||||
|
Build a Lfs1 filesystem on *block_dev*.
|
||||||
|
|
||||||
|
.. class:: VfsLfs2(block_dev)
|
||||||
|
|
||||||
|
Create a filesystem object that uses the `littlefs v2 filesystem format`_.
|
||||||
|
Storage of the littlefs filesystem is provided by *block_dev*, which must
|
||||||
|
support the :ref:`extended interface <block-device-interface>`.
|
||||||
|
Objects created by this constructor can be mounted using :func:`mount`.
|
||||||
|
|
||||||
|
See :ref:`filesystem` for more information.
|
||||||
|
|
||||||
|
.. staticmethod:: mkfs(block_dev)
|
||||||
|
|
||||||
|
Build a Lfs2 filesystem on *block_dev*.
|
||||||
|
|
||||||
|
.. _littlefs v1 filesystem format: https://github.com/ARMmbed/littlefs/tree/v1
|
||||||
|
.. _littlefs v2 filesystem format: https://github.com/ARMmbed/littlefs
|
||||||
|
|
||||||
Block devices
|
Block devices
|
||||||
-------------
|
-------------
|
||||||
|
|
||||||
|
@ -187,9 +216,15 @@ implementation of this class will usually allow access to the memory-like
|
||||||
functionality a piece of hardware (like flash memory). A block device can be
|
functionality a piece of hardware (like flash memory). A block device can be
|
||||||
used by a particular filesystem driver to store the data for its filesystem.
|
used by a particular filesystem driver to store the data for its filesystem.
|
||||||
|
|
||||||
|
.. _block-device-interface:
|
||||||
|
|
||||||
|
Simple and extended interface
|
||||||
|
.............................
|
||||||
|
|
||||||
There are two compatible signatures for the ``readblocks`` and ``writeblocks``
|
There are two compatible signatures for the ``readblocks`` and ``writeblocks``
|
||||||
methods (see below), in order to support a variety of use cases. A given block
|
methods (see below), in order to support a variety of use cases. A given block
|
||||||
device may implement one form or the other, or both at the same time.
|
device may implement one form or the other, or both at the same time. The second
|
||||||
|
form (with the offset parameter) is referred to as the "extended interface".
|
||||||
|
|
||||||
.. class:: AbstractBlockDev(...)
|
.. class:: AbstractBlockDev(...)
|
||||||
|
|
||||||
|
@ -247,64 +282,5 @@ device may implement one form or the other, or both at the same time.
|
||||||
(*arg* is unused)
|
(*arg* is unused)
|
||||||
- 6 -- erase a block, *arg* is the block number to erase
|
- 6 -- erase a block, *arg* is the block number to erase
|
||||||
|
|
||||||
By way of example, the following class will implement a block device that stores
|
See :ref:`filesystem` for example implementations of block devices using both
|
||||||
its data in RAM using a ``bytearray``::
|
protocols.
|
||||||
|
|
||||||
class RAMBlockDev:
|
|
||||||
def __init__(self, block_size, num_blocks):
|
|
||||||
self.block_size = block_size
|
|
||||||
self.data = bytearray(block_size * num_blocks)
|
|
||||||
|
|
||||||
def readblocks(self, block_num, buf):
|
|
||||||
for i in range(len(buf)):
|
|
||||||
buf[i] = self.data[block_num * self.block_size + i]
|
|
||||||
|
|
||||||
def writeblocks(self, block_num, buf):
|
|
||||||
for i in range(len(buf)):
|
|
||||||
self.data[block_num * self.block_size + i] = buf[i]
|
|
||||||
|
|
||||||
def ioctl(self, op, arg):
|
|
||||||
if op == 4: # get number of blocks
|
|
||||||
return len(self.data) // self.block_size
|
|
||||||
if op == 5: # get block size
|
|
||||||
return self.block_size
|
|
||||||
|
|
||||||
It can be used as follows::
|
|
||||||
|
|
||||||
import uos
|
|
||||||
|
|
||||||
bdev = RAMBlockDev(512, 50)
|
|
||||||
uos.VfsFat.mkfs(bdev)
|
|
||||||
vfs = uos.VfsFat(bdev)
|
|
||||||
uos.mount(vfs, '/ramdisk')
|
|
||||||
|
|
||||||
An example of a block device that supports both signatures and behaviours of
|
|
||||||
the :meth:`readblocks` and :meth:`writeblocks` methods is::
|
|
||||||
|
|
||||||
class RAMBlockDev:
|
|
||||||
def __init__(self, block_size, num_blocks):
|
|
||||||
self.block_size = block_size
|
|
||||||
self.data = bytearray(block_size * num_blocks)
|
|
||||||
|
|
||||||
def readblocks(self, block, buf, offset=0):
|
|
||||||
addr = block_num * self.block_size + offset
|
|
||||||
for i in range(len(buf)):
|
|
||||||
buf[i] = self.data[addr + i]
|
|
||||||
|
|
||||||
def writeblocks(self, block_num, buf, offset=None):
|
|
||||||
if offset is None:
|
|
||||||
# do erase, then write
|
|
||||||
for i in range(len(buf) // self.block_size):
|
|
||||||
self.ioctl(6, block_num + i)
|
|
||||||
offset = 0
|
|
||||||
addr = block_num * self.block_size + offset
|
|
||||||
for i in range(len(buf)):
|
|
||||||
self.data[addr + i] = buf[i]
|
|
||||||
|
|
||||||
def ioctl(self, op, arg):
|
|
||||||
if op == 4: # block count
|
|
||||||
return len(self.data) // self.block_size
|
|
||||||
if op == 5: # block size
|
|
||||||
return self.block_size
|
|
||||||
if op == 6: # block erase
|
|
||||||
return 0
|
|
||||||
|
|
|
@ -0,0 +1,278 @@
|
||||||
|
.. _filesystem:
|
||||||
|
|
||||||
|
Working with filesystems
|
||||||
|
========================
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
|
||||||
|
This tutorial describes how MicroPython provides an on-device filesystem,
|
||||||
|
allowing standard Python file I/O methods to be used with persistent storage.
|
||||||
|
|
||||||
|
MicroPython automatically creates a default configuration and auto-detects the
|
||||||
|
primary filesystem, so this tutorial will be mostly useful if you want to modify
|
||||||
|
the partitioning, filesystem type, or use custom block devices.
|
||||||
|
|
||||||
|
The filesystem is typically backed by internal flash memory on the device, but
|
||||||
|
can also use external flash, RAM, or a custom block device.
|
||||||
|
|
||||||
|
On some ports (e.g. STM32), the filesystem may also be available over USB MSC to
|
||||||
|
a host PC. :ref:`pyboard_py` also provides a way for the host PC to access to
|
||||||
|
the filesystem on all ports.
|
||||||
|
|
||||||
|
Note: This is mainly for use on bare-metal ports like STM32 and ESP32. On ports
|
||||||
|
with an operating system (e.g. the Unix port) the filesystem is provided by the
|
||||||
|
host OS.
|
||||||
|
|
||||||
|
VFS
|
||||||
|
---
|
||||||
|
|
||||||
|
MicroPython implements a Unix-like Virtual File System (VFS) layer. All mounted
|
||||||
|
filesystems are combined into a single virtual filesystem, starting at the root
|
||||||
|
``/``. Filesystems are mounted into directories in this structure, and at
|
||||||
|
startup the working directory is changed to where the primary filesystem is
|
||||||
|
mounted.
|
||||||
|
|
||||||
|
On STM32 / Pyboard, the internal flash is mounted at ``/flash``, and optionally
|
||||||
|
the SDCard at ``/sd``. On ESP8266/ESP32, the primary filesystem is mounted at
|
||||||
|
``/``.
|
||||||
|
|
||||||
|
Block devices
|
||||||
|
-------------
|
||||||
|
|
||||||
|
A block device is an instance of a class that implements the
|
||||||
|
:class:`uos.AbstractBlockDev` protocol.
|
||||||
|
|
||||||
|
Built-in block devices
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Ports provide built-in block devices to access their primary flash.
|
||||||
|
|
||||||
|
On power-on, MicroPython will attempt to detect the filesystem on the default
|
||||||
|
flash and configure and mount it automatically. If no filesystem is found,
|
||||||
|
MicroPython will attempt to create a FAT filesystem spanning the entire flash.
|
||||||
|
Ports can also provide a mechanism to "factory reset" the primary flash, usually
|
||||||
|
by some combination of button presses at power on.
|
||||||
|
|
||||||
|
STM32 / Pyboard
|
||||||
|
...............
|
||||||
|
|
||||||
|
The :ref:`pyb.Flash <pyb.Flash>` class provides access to the internal flash. On some
|
||||||
|
boards which have larger external flash (e.g. Pyboard D), it will use that
|
||||||
|
instead. The ``start`` kwarg should always be specified, i.e.
|
||||||
|
``pyb.Flash(start=0)``.
|
||||||
|
|
||||||
|
Note: For backwards compatibility, when constructed with no arguments (i.e.
|
||||||
|
``pyb.Flash()``), it only implements the simple block interface and reflects the
|
||||||
|
virtual device presented to USB MSC (i.e. it includes a virtual partition table
|
||||||
|
at the start).
|
||||||
|
|
||||||
|
ESP8266
|
||||||
|
.......
|
||||||
|
|
||||||
|
The internal flash is exposed as a block device object which is created in the
|
||||||
|
``flashbdev`` module on start up. This object is by default added as a global
|
||||||
|
variable so it can usually be accessed simply as ``bdev``. This implements the
|
||||||
|
extended interface.
|
||||||
|
|
||||||
|
ESP32
|
||||||
|
.....
|
||||||
|
|
||||||
|
The :class:`esp32.Partition` class implements a block device for partitions
|
||||||
|
defined for the board. Like ESP8266, there is a global variable ``bdev`` which
|
||||||
|
points to the default partition. This implements the extended interface.
|
||||||
|
|
||||||
|
Custom block devices
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
The following class implements a simple block device that stores its data in
|
||||||
|
RAM using a ``bytearray``::
|
||||||
|
|
||||||
|
class RAMBlockDev:
|
||||||
|
def __init__(self, block_size, num_blocks):
|
||||||
|
self.block_size = block_size
|
||||||
|
self.data = bytearray(block_size * num_blocks)
|
||||||
|
|
||||||
|
def readblocks(self, block_num, buf):
|
||||||
|
for i in range(len(buf)):
|
||||||
|
buf[i] = self.data[block_num * self.block_size + i]
|
||||||
|
|
||||||
|
def writeblocks(self, block_num, buf):
|
||||||
|
for i in range(len(buf)):
|
||||||
|
self.data[block_num * self.block_size + i] = buf[i]
|
||||||
|
|
||||||
|
def ioctl(self, op, arg):
|
||||||
|
if op == 4: # get number of blocks
|
||||||
|
return len(self.data) // self.block_size
|
||||||
|
if op == 5: # get block size
|
||||||
|
return self.block_size
|
||||||
|
|
||||||
|
It can be used as follows::
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
bdev = RAMBlockDev(512, 50)
|
||||||
|
os.VfsFat.mkfs(bdev)
|
||||||
|
os.mount(bdev, '/ramdisk')
|
||||||
|
|
||||||
|
An example of a block device that supports both the simple and extended
|
||||||
|
interface (i.e. both signatures and behaviours of the
|
||||||
|
:meth:`uos.AbstractBlockDev.readblocks` and
|
||||||
|
:meth:`uos.AbstractBlockDev.writeblocks` methods) is::
|
||||||
|
|
||||||
|
class RAMBlockDev:
|
||||||
|
def __init__(self, block_size, num_blocks):
|
||||||
|
self.block_size = block_size
|
||||||
|
self.data = bytearray(block_size * num_blocks)
|
||||||
|
|
||||||
|
def readblocks(self, block, buf, offset=0):
|
||||||
|
addr = block_num * self.block_size + offset
|
||||||
|
for i in range(len(buf)):
|
||||||
|
buf[i] = self.data[addr + i]
|
||||||
|
|
||||||
|
def writeblocks(self, block_num, buf, offset=None):
|
||||||
|
if offset is None:
|
||||||
|
# do erase, then write
|
||||||
|
for i in range(len(buf) // self.block_size):
|
||||||
|
self.ioctl(6, block_num + i)
|
||||||
|
offset = 0
|
||||||
|
addr = block_num * self.block_size + offset
|
||||||
|
for i in range(len(buf)):
|
||||||
|
self.data[addr + i] = buf[i]
|
||||||
|
|
||||||
|
def ioctl(self, op, arg):
|
||||||
|
if op == 4: # block count
|
||||||
|
return len(self.data) // self.block_size
|
||||||
|
if op == 5: # block size
|
||||||
|
return self.block_size
|
||||||
|
if op == 6: # block erase
|
||||||
|
return 0
|
||||||
|
|
||||||
|
As it supports the extended interface, it can be used with :class:`littlefs
|
||||||
|
<uos.VfsLfs2>`::
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
bdev = RAMBlockDev(512, 50)
|
||||||
|
os.VfsLfs2.mkfs(bdev)
|
||||||
|
os.mount(bdev, '/ramdisk')
|
||||||
|
|
||||||
|
Filesystems
|
||||||
|
-----------
|
||||||
|
|
||||||
|
MicroPython ports can provide implementations of :class:`FAT <uos.VfsFat>`,
|
||||||
|
:class:`littlefs v1 <uos.VfsLfs1>` and :class:`littlefs v2 <uos.VfsLfs2>`.
|
||||||
|
|
||||||
|
The following table shows which filesystems are included in the firmware by
|
||||||
|
default for given port/board combinations, however they can be optionally
|
||||||
|
enabled in a custom firmware build.
|
||||||
|
|
||||||
|
==================== ===== =========== ===========
|
||||||
|
Board FAT littlefs v1 littlefs v2
|
||||||
|
==================== ===== =========== ===========
|
||||||
|
pyboard 1.0, 1.1, D Yes No Yes
|
||||||
|
Other STM32 Yes No No
|
||||||
|
ESP8266 Yes No No
|
||||||
|
ESP32 Yes No Yes
|
||||||
|
==================== ===== =========== ===========
|
||||||
|
|
||||||
|
FAT
|
||||||
|
~~~
|
||||||
|
|
||||||
|
The main advantage of the FAT filesystem is that it can be accessed over USB MSC
|
||||||
|
on supported boards (e.g. STM32) without any additional drivers required on the
|
||||||
|
host PC.
|
||||||
|
|
||||||
|
However, FAT is not tolerant to power failure during writes and this can lead to
|
||||||
|
filesystem corruption. For applications that do not require USB MSC, it is
|
||||||
|
recommended to use littlefs instead.
|
||||||
|
|
||||||
|
To format the entire flash using FAT::
|
||||||
|
|
||||||
|
# ESP8266 and ESP32
|
||||||
|
import os
|
||||||
|
os.umount('/')
|
||||||
|
os.VfsFat.mkfs(bdev)
|
||||||
|
os.mount(bdev, '/')
|
||||||
|
|
||||||
|
# STM32
|
||||||
|
import os, pyb
|
||||||
|
os.umount('/flash')
|
||||||
|
os.VfsFat.mkfs(pyb.Flash(start=0))
|
||||||
|
os.mount(pyb.Flash(start=0), '/flash')
|
||||||
|
os.chdir('/flash')
|
||||||
|
|
||||||
|
Littlefs
|
||||||
|
~~~~~~~~
|
||||||
|
|
||||||
|
Littlefs_ is a filesystem designed for flash-based devices, and is much more
|
||||||
|
resistant to filesystem corruption.
|
||||||
|
|
||||||
|
Note: It can be still be accessed over USB MSC using the `littlefs FUSE
|
||||||
|
driver`_. Note that you must use the ``-b=4096`` option to override the block
|
||||||
|
size.
|
||||||
|
|
||||||
|
.. _littlefs FUSE driver: https://github.com/ARMmbed/littlefs-fuse/tree/master/littlefs
|
||||||
|
|
||||||
|
.. _Littlefs: https://github.com/ARMmbed/littlefs
|
||||||
|
|
||||||
|
To format the entire flash using littlefs v2::
|
||||||
|
|
||||||
|
# ESP8266 and ESP32
|
||||||
|
import os
|
||||||
|
os.umount('/')
|
||||||
|
os.VfsLfs2.mkfs(bdev)
|
||||||
|
os.mount(bdev, '/')
|
||||||
|
|
||||||
|
# STM32
|
||||||
|
import os, pyb
|
||||||
|
os.umount('/flash')
|
||||||
|
os.VfsLfs2.mkfs(pyb.Flash(start=0))
|
||||||
|
os.mount(pyb.Flash(start=0), '/flash')
|
||||||
|
os.chdir('/flash')
|
||||||
|
|
||||||
|
Hybrid (STM32)
|
||||||
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
By using the ``start`` and ``len`` kwargs to :class:`pyb.Flash`, you can create
|
||||||
|
block devices spanning a subset of the flash device.
|
||||||
|
|
||||||
|
For example, to configure the first 256kiB as FAT (and available over USB MSC),
|
||||||
|
and the remainder as littlefs::
|
||||||
|
|
||||||
|
import os, pyb
|
||||||
|
os.umount('/flash')
|
||||||
|
p1 = pyb.Flash(start=0, len=256*1024)
|
||||||
|
p2 = pyb.Flash(start=256*1024)
|
||||||
|
os.VfsFat.mkfs(p1)
|
||||||
|
os.VfsLfs2.mkfs(p2)
|
||||||
|
os.mount(p1, '/flash')
|
||||||
|
os.mount(p2, '/data')
|
||||||
|
os.chdir('/flash')
|
||||||
|
|
||||||
|
This might be useful to make your Python files, configuration and other
|
||||||
|
rarely-modified content available over USB MSC, but allowing for frequently
|
||||||
|
changing application data to reside on littlefs with better resilience to power
|
||||||
|
failure, etc.
|
||||||
|
|
||||||
|
The partition at offset ``0`` will be mounted automatically (and the filesystem
|
||||||
|
type automatically detected), but you can add::
|
||||||
|
|
||||||
|
import os, pyb
|
||||||
|
p2 = pyb.Flash(start=256*1024)
|
||||||
|
os.mount(p2, '/data')
|
||||||
|
|
||||||
|
to ``boot.py`` to mount the data partition.
|
||||||
|
|
||||||
|
Hybrid (ESP32)
|
||||||
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
On ESP32, if you build custom firmware, you can modify ``partitions.csv`` to
|
||||||
|
define an arbitrary partition layout.
|
||||||
|
|
||||||
|
At boot, the partition named "vfs" will be mounted at ``/`` by default, but any
|
||||||
|
additional partitions can be mounted in your ``boot.py`` using::
|
||||||
|
|
||||||
|
import esp32, os
|
||||||
|
p = esp32.Partition.find(esp32.Partition.TYPE_DATA, label='foo')
|
||||||
|
os.mount(p, '/foo')
|
||||||
|
|
|
@ -26,4 +26,5 @@ implementation and the best practices to use them.
|
||||||
constrained.rst
|
constrained.rst
|
||||||
packages.rst
|
packages.rst
|
||||||
asm_thumb2_index.rst
|
asm_thumb2_index.rst
|
||||||
|
filesystem.rst
|
||||||
pyboard.py.rst
|
pyboard.py.rst
|
||||||
|
|
Loading…
Reference in New Issue