fs.walk¶

Machinery for walking a filesystem.

Walking a filesystem means recursively visiting a directory and any sub-directories. It is a fairly common requirement for copying, searching etc. See Walking for details.

class fs.walk.BoundWalker(fs, walker_class=<class 'fs.walk.Walker'>)[source]¶

A class that binds a Walker instance to a FS instance.

Parameters:	fs (FS) – A filesystem instance. walker_class (type) – A `WalkerBase` sub-class. The default uses `Walker`.

You will typically not need to create instances of this class explicitly. Filesystems have a walk property which returns a BoundWalker object.

Example

>>> import fs
>>> home_fs = fs.open_fs('~/')
>>> home_fs.walk
BoundWalker(OSFS('/Users/will', encoding='utf-8'))

A BoundWalker is callable. Calling it is an alias for walk.

__call__(path=u'/', namespaces=None, **kwargs)¶

Walk the directory structure of a filesystem.

Keyword Arguments:
Parameters:	path (str) – namespaces (list, optional) – A list of namespaces to include in the resource information, e.g. `['basic', 'access']` (defaults to `['basic']`).
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter (list) – If supplied, this parameter should be a list of file name patterns, e.g. `['.py']`. Files will only be returned if the final component matches one of the patterns. exclude_dirs* (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	an iterator of `(<path>, <dirs>, <files>)` named tuples, where `<path>` is an absolute path to a directory, and `<dirs>` and `<files>` are a list of `Info` objects for directories and files in `<path>`.
Return type:	Iterator

Example

>>> home_fs = open_fs('~/')
>>> walker = Walker(filter=['*.py'])
>>> for path, dirs, files in walker.walk(home_fs, namespaces=['details']):
...     print("[{}]".format(path))
...     print("{} directories".format(len(dirs)))
...     total = sum(info.size for info in files)
...     print("{} bytes {}".format(total))

This method invokes Walker.walk with bound FS object.

dirs(path=u'/', **kwargs)[source]¶

Walk a filesystem, yielding absolute paths to directories.

Keyword Arguments:
Parameters:	path (str) – A path to a directory.
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	an iterator over directory paths (absolute from the filesystem root).
Return type:	Iterator

This method invokes Walker.dirs with the bound FS object.

files(path=u'/', **kwargs)[source]¶

Walk a filesystem, yielding absolute paths to files.

Keyword Arguments:
Parameters:	path (str) – A path to a directory.
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter (list) – If supplied, this parameter should be a list of file name patterns, e.g. `['.py']`. Files will only be returned if the final component matches one of the patterns. exclude_dirs* (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	An iterator over file paths (absolute from the filesystem root).
Return type:	Iterator

This method invokes Walker.files with the bound FS object.

info(path=u'/', namespaces=None, **kwargs)[source]¶

Walk a filesystem, yielding path and Info of resources.

Keyword Arguments:
Parameters:	path (str) – A path to a directory. namespaces (list, optional) – A list of namespaces to include in the resource information, e.g. `['basic', 'access']` (defaults to `['basic']`).
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter (list) – If supplied, this parameter should be a list of file name patterns, e.g. `['.py']`. Files will only be returned if the final component matches one of the patterns. exclude_dirs* (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	an iterable yielding tuples of `(<absolute path>, <resource info>)`.
Return type:	Iterable

This method invokes Walker.info with the bound FS object.

walk(path=u'/', namespaces=None, **kwargs)[source]¶

Walk the directory structure of a filesystem.

Keyword Arguments:
Parameters:	path (str) – namespaces (list, optional) – A list of namespaces to include in the resource information, e.g. `['basic', 'access']` (defaults to `['basic']`).
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter (list) – If supplied, this parameter should be a list of file name patterns, e.g. `['.py']`. Files will only be returned if the final component matches one of the patterns. exclude_dirs* (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	an iterator of `(<path>, <dirs>, <files>)` named tuples, where `<path>` is an absolute path to a directory, and `<dirs>` and `<files>` are a list of `Info` objects for directories and files in `<path>`.
Return type:	Iterator

Example

>>> home_fs = open_fs('~/')
>>> walker = Walker(filter=['*.py'])
>>> for path, dirs, files in walker.walk(home_fs, namespaces=['details']):
...     print("[{}]".format(path))
...     print("{} directories".format(len(dirs)))
...     total = sum(info.size for info in files)
...     print("{} bytes {}".format(total))

This method invokes Walker.walk with bound FS object.

class fs.walk.Step(path, dirs, files)¶

type: a step in a directory walk.

__getnewargs__()¶: Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶: Exclude the OrderedDict from pickling

__repr__()¶: Return a nicely formatted representation string

dirs¶: Alias for field number 1

files¶: Alias for field number 2

path¶: Alias for field number 0

class fs.walk.Walker(ignore_errors=False, on_error=None, search=u'breadth', filter=None, exclude_dirs=None, max_depth=None)[source]¶

A walker object recursively lists directories in a filesystem.

Parameters:

ignore_errors (bool) – If True, any errors reading a directory will be ignored, otherwise exceptions will be raised.
on_error (callable, optional) – If ignore_errors is False, then this callable will be invoked for a path and the exception object. It should return True to ignore the error, or False to re-raise it.
search (str) – If 'breadth' then the directory will be walked top down. Set to 'depth' to walk bottom up.
filter (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g. ['*.py']. Files will only be returned if the final component matches one of the patterns.
exclude_dirs (list, optional) – A list of patterns that will be used to filter out directories from the walk. e.g. ['*.svn', '*.git'].
max_depth (int, optional) – Maximum directory depth to walk.

classmethod bind(fs)[source]¶

Bind a Walker instance to a given filesystem.

This binds in instance of the Walker to a given filesystem, so that you won’t need to explicitly provide the filesystem as a parameter.

Parameters:	fs (FS) – A filesystem object.
Returns:	a bound walker.
Return type:	BoundWalker

Example

>>> from fs import open_fs
>>> from fs.walk import Walker
>>> home_fs = open_fs('~/')
>>> walker = Walker.bind(home_fs)
>>> for path in walker.files(filter=['*.py']):
...     print(path)

Unless you have written a customized walker class, you will be unlikely to need to call this explicitly, as filesystem objects already have a walk attribute which is a bound walker object.

Example

>>> from fs import open_fs
>>> home_fs = open_fs('~/')
>>> for path in home_fs.walk.files(filter=['*.py']):
...     print(path)

check_file(fs, info)[source]¶

Check if a filename should be included.

Override to exclude files from the walk.

Parameters:	fs (FS) – A filesystem instance. info (Info) – A resource info object.
Returns:	`True` if the file should be included.
Return type:	bool

check_open_dir(fs, path, info)[source]¶

Check if a directory should be opened.

Override to exclude directories from the walk.

Parameters:	fs (FS) – A filesystem instance. path (str) – Path to directory. info (Info) – A resource info object for the directory.
Returns:	`True` if the directory should be opened.
Return type:	bool

check_scan_dir(fs, path, info)[source]¶

Check if a directory should be scanned.

Override to omit scanning of certain directories. If a directory is omitted, it will appear in the walk but its files and sub-directories will not.

Parameters:	fs (FS) – A filesystem instance. path (str) – Path to directory. info (Info) – A resource info object for the directory.
Returns:	`True` if the directory should be scanned.
Return type:	bool

dirs(fs, path=u'/')[source]¶

Walk a filesystem, yielding absolute paths to directories.

Parameters:	fs (FS) – A filesystem instance. path (str) – A path to a directory on the filesystem.
Yields:	str – absolute path to directories on the filesystem found recursively within the given directory.

files(fs, path=u'/')[source]¶

Walk a filesystem, yielding absolute paths to files.

Parameters:	fs (FS) – A filesystem instance. path (str) – A path to a directory on the filesystem.
Yields:	str – absolute path to files on the filesystem found recursively within the given directory.

info(fs, path=u'/', namespaces=None)[source]¶

Walk a filesystem, yielding tuples of (<path>, <info>).

Parameters:	fs (FS) – A filesystem instance. path (str) – A path to a directory on the filesystem. namespaces (list, optional) – A list of additional namespaces to add to the `Info` objects.
Yields:	(str, Info) – a tuple of `(<absolute path>, <resource info>)`.

walk(fs, path=u'/', namespaces=None)[source]¶

Walk the directory structure of a filesystem.

Parameters:	fs (FS) – A filesystem instance. path (str) – A path to a directory on the filesystem. namespaces (list, optional) – A list of additional namespaces to add to the `Info` objects.
Returns:	an iterator of `Step` instances.
Return type:	collections.Iterator

The return value is an iterator of (<path>, <dirs>, <files>) named tuples, where <path> is an absolute path to a directory, and <dirs> and <files> are a list of Info objects for directories and files in <path>.

Example

>>> home_fs = open_fs('~/')
>>> walker = Walker(filter=['*.py'])
>>> namespaces = ['details']
>>> for path, dirs, files in walker.walk(home_fs, namespaces)
...    print("[{}]".format(path))
...    print("{} directories".format(len(dirs)))
...    total = sum(info.size for info in files)
...    print("{} bytes {}".format(total))