Package plumbum.machines

class plumbum.machines.env.BaseEnv(path_factory, pathsep)[source]

The base class of LocalEnv and RemoteEnv

__call__(*args, **kwds)[source]

A context manager that can be used for temporal modifications of the environment. Any time you enter the context, a copy of the old environment is stored, and then restored, when the context exits.

Parameters:
  • args – Any positional arguments for update()
  • kwargs – Any keyword arguments for update()
__iter__()[source]

Returns an iterator over the items (key, value) of current environment (like dict.items)

__len__()[source]

Returns the number of elements of the current environment

__contains__(name)[source]

Tests whether an environment variable exists in the current environment

__getitem__(name)[source]

Returns the value of the given environment variable from current environment, raising a KeyError if it does not exist

keys()[source]

Returns the keys of the current environment (like dict.keys)

items()[source]

Returns the items of the current environment (like dict.items)

values()[source]

Returns the values of the current environment (like dict.values)

get(name, *default)[source]

Returns the keys of the current environment (like dict.keys)

__delitem__(name)[source]

Deletes an environment variable from the current environment

__setitem__(name, value)[source]

Sets/replaces an environment variable’s value in the current environment

pop(name, *default)[source]

Pops an element from the current environment (like dict.pop)

clear()[source]

Clears the current environment (like dict.clear)

update(*args, **kwargs)[source]

Updates the current environment (like dict.update)

getdict()[source]

Returns the environment as a real dictionary

path

The system’s PATH (as an easy-to-manipulate list)

home

Get or set the home path

user

Return the user name, or None if it is not set

class plumbum.machines.local.LocalEnv[source]

The local machine’s environment; exposes a dict-like interface

expand(expr)[source]

Expands any environment variables and home shortcuts found in expr (like os.path.expanduser combined with os.path.expandvars)

Parameters:expr – An expression containing environment variables (as $FOO) or home shortcuts (as ~/.bashrc)
Returns:The expanded string
expanduser(expr)[source]

Expand home shortcuts (e.g., ~/foo/bar or ~john/foo/bar)

Parameters:expr – An expression containing home shortcuts
Returns:The expanded string
class plumbum.machines.local.LocalMachine[source]

The local machine (a singleton object). It serves as an entry point to everything related to the local machine, such as working directory and environment manipulation, command creation, etc.

Attributes:

  • cwd - the local working directory
  • env - the local environment
  • encoding - the local machine’s default encoding (sys.getfilesystemencoding())
classmethod which(progname)[source]

Looks up a program in the PATH. If the program is not found, raises CommandNotFound

Parameters:progname – The program’s name. Note that if underscores (_) are present in the name, and the exact name is not found, they will be replaced in turn by hyphens (-) then periods (.), and the name will be looked up again for each alternative
Returns:A LocalPath
path(*parts)[source]

A factory for LocalPaths. Usage: p = local.path("/usr", "lib", "python2.7")

__getitem__(cmd)[source]

Returns a Command object representing the given program. cmd can be a string or a LocalPath; if it is a path, a command representing this path will be returned; otherwise, the program name will be looked up in the system’s PATH (using which). Usage:

ls = local["ls"]
__contains__(cmd)[source]

Tests for the existance of the command, e.g., "ls" in plumbum.local. cmd can be anything acceptable by __getitem__.

daemonic_popen(command, cwd='/')[source]

On POSIX systems:

Run command as a UNIX daemon: fork a child process to setpid, redirect std handles to /dev/null, umask, close all fds, chdir to cwd, then fork and exec command. Returns a Popen process that can be used to poll/wait for the executed command (but keep in mind that you cannot access std handles)

On Windows:

Run command as a “Windows daemon”: detach from controlling console and create a new process group. This means that the command will not receive console events and would survive its parent’s termination. Returns a Popen object.

Note

this does not run command as a system service, only detaches it from its parent.

New in version 1.3.

list_processes()[source]

Returns information about all running processes (on POSIX systems: using ps)

New in version 1.3.

pgrep(pattern)[source]

Process grep: return information about all processes whose command-line args match the given regex pattern

session(new_session=False)[source]

Creates a new ShellSession object; this invokes /bin/sh and executes commands on it over stdin/stdout/stderr

tempdir(*args, **kwds)[source]

A context manager that creates a temporary directory, which is removed when the context exits

__weakref__

list of weak references to the object (if defined)

as_user(*args, **kwds)[source]

Run nested commands as the given user. For example:

head = local["head"]
head("-n1", "/dev/sda1")    # this will fail...
with local.as_user():
    head("-n1", "/dev/sda1")
Parameters:username – The user to run commands as. If not given, root (or Administrator) is assumed
as_root()[source]

A shorthand for as_user("root")

python = LocalCommand('/usr/bin/python')

A command that represents the current python interpreter (sys.executable)

plumbum.machines.local.local = <plumbum.machines.local.LocalMachine object>

The local machine (a singleton object). It serves as an entry point to everything related to the local machine, such as working directory and environment manipulation, command creation, etc.

Attributes:

  • cwd - the local working directory
  • env - the local environment
  • encoding - the local machine’s default encoding (sys.getfilesystemencoding())
exception plumbum.machines.session.ShellSessionError[source]

Raises when something goes wrong when calling ShellSession.popen

__weakref__

list of weak references to the object (if defined)

class plumbum.machines.session.MarkedPipe(pipe, marker)[source]

A pipe-like object from which you can read lines; the pipe will return report EOF (the empty string) when a special marker is detected

close()[source]

‘Closes’ the marked pipe; following calls to readline will return

readline()[source]

Reads the next line from the pipe; returns “” when the special marker is reached. Raises EOFError if the underlying pipe has closed

class plumbum.machines.session.SessionPopen(argv, isatty, stdin, stdout, stderr, encoding)[source]

A shell-session-based Popen-like object (has the following attributes: stdin, stdout, stderr, returncode)

poll()[source]

Returns the process’ exit code or None if it’s still running

wait()[source]

Waits for the process to terminate and returns its exit code

communicate(input=None)[source]

Consumes the process’ stdout and stderr until the it terminates.

Parameters:input – An optional bytes/buffer object to send to the process over stdin
Returns:A tuple of (stdout, stderr)
__weakref__

list of weak references to the object (if defined)

class plumbum.machines.session.ShellSession(proc, encoding='auto', isatty=False, connect_timeout=5)[source]

An abstraction layer over shell sessions. A shell session is the execution of an interactive shell (/bin/sh or something compatible), over which you may run commands (sent over stdin). The output of is then read from stdout and stderr. Shell sessions are less “robust” than executing a process on its own, and they are susseptible to all sorts of malformatted-strings attacks, and there is little benefit from using them locally. However, they can greatly speed up remote connections, and are required for the implementation of SshMachine, as they allow us to send multiple commands over a single SSH connection (setting up separate SSH connections incurs a high overhead). Try to avoid using shell sessions, unless you know what you’re doing.

Instances of this class may be used as context-managers.

Parameters:
  • proc – The underlying shell process (with open stdin, stdout and stderr)
  • encoding – The encoding to use for the shell session. If "auto", the underlying process’ encoding is used.
  • isatty – If true, assume the shell has a TTY and that stdout and stderr are unified
  • connect_timeout – The timeout to connect to the shell, after which, if no prompt is seen, the shell process is killed
alive()[source]

Returns True if the underlying shell process is alive, False otherwise

close()[source]

Closes (terminates) the shell session

popen(cmd)[source]

Runs the given command in the shell, adding some decoration around it. Only a single command can be executed at any given time.

Parameters:cmd – The command (string or Command object) to run
Returns:A SessionPopen instance
__weakref__

list of weak references to the object (if defined)

run(cmd, retcode=0)[source]

Runs the given command

Parameters:
  • cmd – The command (string or Command object) to run
  • retcode – The expected return code (0 by default). Set to None in order to ignore erroneous return codes
Returns:

A tuple of (return code, stdout, stderr)

Remote Machines

class plumbum.machines.remote.RemoteEnv(remote)[source]

The remote machine’s environment; exposes a dict-like interface

__delitem__(name)[source]

Deletes an environment variable from the current environment

__setitem__(name, value)[source]

Sets/replaces an environment variable’s value in the current environment

pop(name, *default)[source]

Pops an element from the current environment (like dict.pop)

update(*args, **kwargs)[source]

Updates the current environment (like dict.update)

expand(expr)[source]

Expands any environment variables and home shortcuts found in expr (like os.path.expanduser combined with os.path.expandvars)

Parameters:expr – An expression containing environment variables (as $FOO) or home shortcuts (as ~/.bashrc)
Returns:The expanded string
expanduser(expr)[source]

Expand home shortcuts (e.g., ~/foo/bar or ~john/foo/bar)

Parameters:expr – An expression containing home shortcuts
Returns:The expanded string
getdelta()[source]

Returns the difference between the this environment and the original environment of the remote machine

class plumbum.machines.remote.BaseRemoteMachine(encoding='utf8', connect_timeout=10, new_session=False)[source]

Represents a remote machine; serves as an entry point to everything related to that remote machine, such as working directory and environment manipulation, command creation, etc.

Attributes:

  • cwd - the remote working directory
  • env - the remote environment
  • encoding - the remote machine’s default encoding (assumed to be UTF8)
  • connect_timeout - the connection timeout
close()[source]

closes the connection to the remote machine; all paths and programs will become defunct

path(*parts)[source]

A factory for RemotePaths. Usage: p = rem.path("/usr", "lib", "python2.7")

which(progname)[source]

Looks up a program in the PATH. If the program is not found, raises CommandNotFound

Parameters:progname – The program’s name. Note that if underscores (_) are present in the name, and the exact name is not found, they will be replaced in turn by hyphens (-) then periods (.), and the name will be looked up again for each alternative
Returns:A RemotePath
__getitem__(cmd)[source]

Returns a Command object representing the given program. cmd can be a string or a RemotePath; if it is a path, a command representing this path will be returned; otherwise, the program name will be looked up in the system’s PATH (using which). Usage:

r_ls = rem["ls"]
__contains__(cmd)[source]

Tests for the existance of the command, e.g., "ls" in remote_machine. cmd can be anything acceptable by __getitem__.

python

A command that represents the default remote python interpreter

session(isatty=False, new_session=False)[source]

Creates a new ShellSession object; this invokes the user’s shell on the remote machine and executes commands on it over stdin/stdout/stderr

download(src, dst)[source]

Downloads a remote file/directory (src) to a local destination (dst). src must be a string or a RemotePath pointing to this remote machine, and dst must be a string or a LocalPath

upload(src, dst)[source]

Uploads a local file/directory (src) to a remote destination (dst). src must be a string or a LocalPath, and dst must be a string or a RemotePath pointing to this remote machine

popen(args, **kwargs)[source]

Spawns the given command on the remote machine, returning a Popen-like object; do not use this method directly, unless you need “low-level” control on the remote process

list_processes()[source]

Returns information about all running processes (on POSIX systems: using ps)

New in version 1.3.

pgrep(pattern)[source]

Process grep: return information about all processes whose command-line args match the given regex pattern

tempdir(*args, **kwds)[source]

A context manager that creates a remote temporary directory, which is removed when the context exits

__weakref__

list of weak references to the object (if defined)

class plumbum.machines.ssh_machine.SshTunnel(session)[source]

An object representing an SSH tunnel (created by SshMachine.tunnel)

close()[source]

Closes(terminates) the tunnel

class plumbum.machines.ssh_machine.SshMachine(host, user=None, port=None, keyfile=None, ssh_command=None, scp_command=None, ssh_opts=(), scp_opts=(), password=None, encoding='utf8', connect_timeout=10, new_session=False)[source]

An implementation of remote machine over SSH. Invoking a remote command translates to invoking it over SSH

with SshMachine("yourhostname") as rem:
    r_ls = rem["ls"]
    # r_ls is the remote `ls`
    # executing r_ls() translates to `ssh yourhostname ls`
Parameters:
  • host – the host name to connect to (SSH server)
  • user – the user to connect as (if None, the default will be used)
  • port – the server’s port (if None, the default will be used)
  • keyfile – the path to the identity file (if None, the default will be used)
  • ssh_command – the ssh command to use; this has to be a Command object; if None, the default ssh client will be used.
  • scp_command – the scp command to use; this has to be a Command object; if None, the default scp program will be used.
  • ssh_opts – any additional options for ssh (a list of strings)
  • scp_opts – any additional options for scp (a list of strings)
  • password – the password to use; requires sshpass be installed. Cannot be used in conjunction with ssh_command or scp_command (will be ignored). NOTE: THIS IS A SECURITY RISK!
  • encoding – the remote machine’s encoding (defaults to UTF8)
  • connect_timeout – specify a connection timeout (the time until shell prompt is seen). The default is 10 seconds. Set to None to disable
  • new_session – whether or not to start the background session as a new session leader (setsid). This will prevent it from being killed on Ctrl+C (SIGINT)
popen(args, ssh_opts=(), **kwargs)[source]

Spawns the given command on the remote machine, returning a Popen-like object; do not use this method directly, unless you need “low-level” control on the remote process

nohup(command)[source]

Runs the given command using nohup and redirects std handles to /dev/null, allowing the command to run “detached” from its controlling TTY or parent. Does not return anything.

session(isatty=False, new_session=False)[source]

Creates a new ShellSession object; this invokes the user’s shell on the remote machine and executes commands on it over stdin/stdout/stderr

tunnel(lport, dport, lhost='localhost', dhost='localhost', connect_timeout=5)[source]

Creates an SSH tunnel from the TCP port (lport) of the local machine (lhost, defaults to "localhost", but it can be any IP you can bind()) to the remote TCP port (dport) of the destination machine (dhost, defaults to "localhost", which means this remote machine). The returned SshTunnel object can be used as a context-manager.

The more conventional use case is the following:

+---------+          +---------+
| Your    |          | Remote  |
| Machine |          | Machine |
+----o----+          +---- ----+
     |                    ^
     |                    |
   lport                dport
     |                    |
     \______SSH TUNNEL____/
            (secure)

Here, you wish to communicate safely between port lport of your machine and port dport of the remote machine. Communication is tunneled over SSH, so the connection is authenticated and encrypted.

The more general case is shown below (where dport != "localhost"):

+---------+          +-------------+      +-------------+
| Your    |          | Remote      |      | Destination |
| Machine |          | Machine     |      | Machine     |
+----o----+          +---- ----o---+      +---- --------+
     |                    ^    |               ^
     |                    |    |               |
lhost:lport               |    |          dhost:dport
     |                    |    |               |
     \_____SSH TUNNEL_____/    \_____SOCKET____/
            (secure)              (not secure)

Usage:

rem = SshMachine("megazord")

with rem.tunnel(1234, 5678):
    sock = socket.socket()
    sock.connect(("localhost", 1234))
    # sock is now tunneled to megazord:5678
download(src, dst)[source]

Downloads a remote file/directory (src) to a local destination (dst). src must be a string or a RemotePath pointing to this remote machine, and dst must be a string or a LocalPath

upload(src, dst)[source]

Uploads a local file/directory (src) to a remote destination (dst). src must be a string or a LocalPath, and dst must be a string or a RemotePath pointing to this remote machine

class plumbum.machines.ssh_machine.PuttyMachine(host, user=None, port=None, keyfile=None, ssh_command=None, scp_command=None, ssh_opts=(), scp_opts=(), encoding='utf8', connect_timeout=10, new_session=False)[source]

PuTTY-flavored SSH connection. The programs plink and pscp are expected to be in the path (or you may provide your own ssh_command and scp_command)

Arguments are the same as for plumbum.machines.remote.SshMachine

session(isatty=False, new_session=False)[source]

Creates a new ShellSession object; this invokes the user’s shell on the remote machine and executes commands on it over stdin/stdout/stderr

class plumbum.machines.paramiko_machine.ParamikoMachine(host, user=None, port=None, password=None, keyfile=None, load_system_host_keys=True, missing_host_policy=None, encoding='utf8', look_for_keys=None, connect_timeout=None)[source]

An implementation of remote machine over Paramiko (a Python implementation of openSSH2 client/server). Invoking a remote command translates to invoking it over SSH

with ParamikoMachine("yourhostname") as rem:
    r_ls = rem["ls"]
    # r_ls is the remote `ls`
    # executing r_ls() is equivalent to `ssh yourhostname ls`, only without
    # spawning a new ssh client
Parameters:
  • host – the host name to connect to (SSH server)
  • user – the user to connect as (if None, the default will be used)
  • port – the server’s port (if None, the default will be used)
  • password – the user’s password (if a password-based authentication is to be performed) (if None, key-based authentication will be used)
  • keyfile – the path to the identity file (if None, the default will be used)
  • load_system_host_keys – whether or not to load the system’s host keys (from /etc/ssh and ~/.ssh). The default is True, which means Paramiko behaves much like the ssh command-line client
  • missing_host_policy – the value passed to the underlying set_missing_host_key_policy of the client. The default is None, which means set_missing_host_key_policy is not invoked and paramiko’s default behavior (reject) is employed
  • encoding – the remote machine’s encoding (defaults to UTF8)
  • look_for_keys – set to False to disable searching for discoverable private key files in ~/.ssh
  • connect_timeout – timeout for TCP connection
sftp

Returns an SFTP client on top of the current SSH connection; it can be used to manipulate files directly, much like an interactive FTP/SFTP session

session(isatty=False, term='vt100', width=80, height=24, new_session=False)[source]

Creates a new ShellSession object; this invokes the user’s shell on the remote machine and executes commands on it over stdin/stdout/stderr

popen(args, stdin=None, stdout=None, stderr=None, new_session=False)[source]

Spawns the given command on the remote machine, returning a Popen-like object; do not use this method directly, unless you need “low-level” control on the remote process

download(src, dst)[source]

Downloads a remote file/directory (src) to a local destination (dst). src must be a string or a RemotePath pointing to this remote machine, and dst must be a string or a LocalPath

upload(src, dst)[source]

Uploads a local file/directory (src) to a remote destination (dst). src must be a string or a LocalPath, and dst must be a string or a RemotePath pointing to this remote machine

connect_sock(dport, dhost='localhost', ipv6=False)[source]

Returns a Paramiko Channel, connected to dhost:dport on the remote machine. The Channel behaves like a regular socket; you can send and recv on it and the data will pass encrypted over SSH. Usage:

mach = ParamikoMachine("myhost")
sock = mach.connect_sock(12345)
data = sock.recv(100)
sock.send("foobar")
sock.close()