Runtime

class mosaic.runtime.runtime.Runtime(**kwargs)

Bases: BaseRPC

Class representing a local runtime of any possible type.

The runtime handles the mosaic life cycle:

• it handles the comms manager, the event loop, the logger and keeps proxies to existing remote runtimes;

• it keeps track of resident mosaic objects (tessera, task) and proxies to those;

• it routes remote commands to these resident mosaic objects.

For referece on accepted parameters, check mosaic.init.

property address

IP address of the runtime.

async_for(*iterables, **kwargs)

async barrier(timeout=None)

Wait until all pending tasks are done. If no timeout is provided, the barrier will wait indefinitely.

Parameters:

timeout (float, optional) –

cmd(sender_id, cmd)

Process incoming command address to one of the resident objects.

Parameters:

• sender_id (str) –

• cmd (CMD) –

connect(sender_id, uid, address, port)

Connect to a specific remote runtime.

Parameters:

• sender_id (str) –

• uid (str) –

• address (str) –

• port (int) –

cpu_load()

CPU load of this runtime as a percentage.

Returns:

CPU load.

Return type:

float

dec_committed_mem(nbytes)

dec_pending_tasks()

dec_ref(sender_id, uid, type)

Decrease reference count for a resident object.

If reference count decreases below 1, deregister the object.

Parameters:

• sender_id (str) – Caller UID.

• uid (str) – UID of the object being referenced.

• type (str) – Type of the object being referenced.

dec_refs(sender_id, msgs)

Decrease reference count for multiple resident objects.

If reference count decreases below 1, deregister the object.

Parameters:

• sender_id (str) – Caller UID.

• msgs (list) – UIDs of the object being referenced.

dec_running_tasks()

deregister(obj)

Deregister CMD object from runtime.

Parameters:

obj (BaseCMD) –

disconnect(sender_id, uid)

Disconnect specific remote runtime.

Parameters:

• sender_id (str) –

• uid (str) –

async drop(uid)

Delete an object from the warehouse.

Parameters:

uid –

fits_in_memory(nbytes)

async get(uid)

Retrieve an object from the warehouse.

Parameters:

uid –

get_comms()

Access comms.

get_event_loop(asyncio_loop=None)

Access event loop.

Parameters:

asyncio_loop (object, optional) – Async loop to use in our mosaic event loop, defaults to new loop.

get_head()

Access head runtime.

get_local_warehouse()

Access local warehouse.

get_monitor()

Access monitor runtime.

get_node(uid=None)

Access specific node runtime.

Parameters:

uid (str) –

get_nodes()

Access all node runtimes.

get_warehouse()

Access warehouse.

get_worker(uid=None)

Access specific worker runtime.

Parameters:

uid (str) –

get_workers()

Access all worker runtimes.

get_zmq_context()

Access ZMQ socket context.

hand(sender_id, address, port)

Handle incoming handshake petition.

Parameters:

• sender_id (str) –

• address (str) –

• port (int) –

inc_committed_mem(nbytes)

inc_pending_tasks()

inc_ref(sender_id, uid, type)

Increase reference count for a resident object.

Parameters:

• sender_id (str) – Caller UID.

• uid (str) – UID of the object being referenced.

• type (str) – Type of the object being referenced.

inc_running_tasks()

async init(**kwargs)

Asynchronous counterpart of __init__.

Parameters:

kwargs –

async init_task(sender_id, task, uid)

Create new task for a tessera in this worker.

Parameters:

• sender_id (str) – Caller UID.

• task (dict) – Task configuration.

• uid (str) – UID of the new task.

async init_tessera(sender_id, cls, uid, args, **kwargs)

Create tessera in this worker.

Parameters:

• sender_id (str) – Caller UID.

• cls (type) – Class of the tessera.

• uid (str) – UID of the new tessera.

• args (tuple, optional) – Arguments for the initialisation of the tessera.

• kwargs (optional) – Keyword arguments for the initialisation of the tessera.

async init_warehouse(**kwargs)

Init warehouse process.

Parameters:

kwargs –

is_head = False

is_monitor = False

is_node = False

is_warehouse = False

is_worker = False

log_debug(sender_id, buf)

Log remote message from sender_id on debug stream.

Parameters:

• sender_id (str) –

• buf (str) –

log_error(sender_id, buf)

Log remote message from sender_id on error stream.

Parameters:

• sender_id (str) –

• buf (str) –

log_info(sender_id, buf)

Log remote message from sender_id on info stream.

Parameters:

• sender_id (str) –

• buf (str) –

log_perf(sender_id, buf)

Log remote message from sender_id on perf stream.

Parameters:

• sender_id (str) –

• buf (str) –

log_warning(sender_id, buf)

Log remote message from sender_id on warning stream.

Parameters:

• sender_id (str) –

• buf (str) –

async maintenance()

Task handling maintenance processes such as object deallocation.

maintenance_msg(method, msg)

Add message to maintenance queue

Parameters:

• method (callable) –

• msg (dict) –

maintenance_queue(fun)

Add callable to maintenance queue

Parameters:

fun (callable) –

memory_limit()

Amount of RSS memory available to the runtime.

Returns:

RSS memory.

Return type:

float

needs_registering(obj_type, obj_uid)

property nodes

Nodes on the network.

property num_nodes

Number of nodes on the network.

property num_workers

Number of workers on the network.

property port

Port of the runtime.

static proxy(name=None, indices=(), uid=None)

Generate proxy from name, indices or UID.

Parameters:

• name (str, optional) –

• indices (tuple, optional) –

• uid (str, optional) –

proxy_from_uid(uid, proxy=None)

Generate a proxy from a UID.

Parameters:

• uid (str) –

• proxy (BaseProxy) –

Return type:

BaseProxy

ps_process

property pubsub_port

Pub-sub port of the runtime.

async put(obj, publish=False, reply=False)

Put an object into the warehouse.

Parameters:

• obj –

• publish –

• reply –

raise_exception(sender_id, exc)

Raise remote exception that ocurred on sender_id.

Parameters:

• sender_id (str) –

• exc (Exception description) –

recv_profile(sender_id, profiler_update)

Process a profiler update.

Parameters:

• sender_id (str) –

• profiler_update (dict) –

register(obj)

Register CMD object with runtime.

Parameters:

obj (BaseCMD) –

remove_proxy_from_uid(uid, proxy=None)

Remove a proxy from a UID.

Parameters:

• uid (str) –

• proxy (BaseProxy) –

request_profile(sender_id)

Return a profiler update.

Parameters:

sender_id (str) –

async send_profile()

Send profiler update to monitor.

set_comms(address=None, port=None)

Set up comms manager.

Parameters:

• address (str, optional) – Address to use, defaults to None. If None, the comms will try to guess the address.

• port (int, optional) – Port to use, defaults to None. If None, the comms will test ports until one becomes available.

set_logger()

Set up logging.

set_profiler()

Set up profiling.

shake(sender_id, network)

Handle handshake response.

Parameters:

• sender_id (str) –

• network (dict) –

async stop(sender_id=None)

Stop runtime.

Parameters:

sender_id (str) –

wait(wait=False)

Wait on the comms loop until done.

Parameters:

wait (bool) – Whether or not to wait, defaults to False.

property workers

Workers on the network.

class mosaic.runtime.runtime.RuntimeProxy(name=None, indices=(), uid=None, comms=None)

Bases: BaseRPC

This class represents a proxy to a remote running runtime.

This proxy can be used to execute methods and commands on the remote runtime simply by calling methods on it.

The proxy uses the comms to direct messages to the correct endpoint using its UID.

Parameters:

• name (str, optional) – Name of the runtime, defaults to None. If no name is provided, the UID has to be given.

• indices (tuple or int, optional) – Indices associated with the runtime, defaults to none.

• uid (str) – UID from which to find the name and indices, defaults to None.

• comms (CommsManager) – Comms instance to use, defaults to global comms.

property address

Remote runtime IP address.

property comms

property port

Remote runtime port.

property pubsub_port

Remote pub-sub port.

property subprocess

Subprocess on which remote runtime lives, if any.