The jupyter_rfb guide¶
Installation¶
Install with pip:
pip install -U jupyter_rfb
Or to install into a conda environment:
conda install -c conda-forge jupyter-rfb
For better performance, also install simplejpeg
or pillow
.
If you plan to hack on this library, see the contributor guide for a dev installation and more.
Subclassing the widget¶
The provided RemoteFrameBuffer
class cannot do much by itself, but it provides
a basis for widgets that want to generate images at the server, and be informed
of user events. The way to use jupyter_rfb
is therefore to create a subclass
and implement two specific methods.
The first method to implement is .get_frame()
, which should return a uint8 numpy array. For example:
class MyRemoteFrameBuffer(jupyter_rfb.RemoteFrameBuffer):
def get_frame(self):
return np.random.uniform(0, 255, (100,100)).astype(np.uint8)
The second method to implement is .handle_event()
,
which accepts an event object. This is where you can react to changes
and user interactions. The most important one may be the resize event,
so that you can match the array size to the region on screen. For
example:
class MyRemoteFrameBuffer(jupyter_rfb.RemoteFrameBuffer):
def handle_event(self, event):
event_type = event["event_type"]
if event_type == "resize":
self.logical_size = event["width"], event["height"]
self.pixel_ratio = event["pixel_ratio"]
elif event_type == "pointer_down":
pass # ...
Logical vs physical pixels¶
The size of e.g. the resize event is expressed in logical pixels. This is a unit of measurement that changes as the user changes the browser zoom level.
By multiplying the logical size with the pixel-ratio, you obtain the physical size, which represents the actual pixels of the screen. With a zoom level of 100% the pixel-ratio is 1 on a regular display and 2 on a HiDPI display, although the actual values may also be affected by the OS’s zoom level.
Scheduling draws¶
The .get_frame()
method is called automatically when a new draw is
performed. There are cases when the widget knows that a redraw is
(probably) required, such as when the widget is resized.
If you want to trigger a redraw (e.g. because certain state has
changed in reaction to user interaction), you can call
.request_draw()
to schedule a new draw.
The scheduling of draws is done in such a way to avoid images being
produced faster than the client can consume them - a process known as
throttling. In more detail: the client sends a confirmation for each
frame that it receives, and the server waits with producing a new frame
until the client has confirmed receiving the nth latest frame. This
mechanism causes the calls to .get_frame()
to match the speed by which
the frames can be communicated and displayed. This helps minimize the
lag and optimize the FPS.
Event throttling¶
Events go from the client (browser) to the server (Python). Some of these are throttled so they are emitted a maximum number of times per second. This is to avoid spamming the communication channel and server process. The throttling applies to the resize, scroll, and pointer_move events.
Taking snapshots¶
In a notebook, the .snapshot()
method can be used to create a picture of the current state of the
widget. This image remains visible when the notebook is in off-line
mode (e.g. in nbviewer). This functionality can be convenient if you’re
using a notebook to tell a story, and you want to display a certain
result that is still visible in off-line mode.
When a widget is first displayed, it automatically creates a snapshot, which is hidden by default, but becomes visible when the widget itself is not loaded. In other words, example notebooks have pretty pictures!
Exceptions and logging¶
The .handle_event()
and .get_frame()
methods are called from a Jupyter
COM event and in an asyncio task, respectively. Under these circumstances,
Jupyter Lab/Notebook may swallow exceptions as well as writes to stdout/stderr.
See issue #35 for details.
These are limitation of Jupyter, and we should expect these to be fixed/improved in the future.
In jupyter_rfb we take measures to make exceptions raised in
either of these methods result in a traceback shown right above the
widget. To ensure that calls to print()
in these methods are also
shown, use self.print()
instead.
Note that any other streaming to stdout and stderr (e.g. logging) may not become visible anywhere.
Measuring statistics¶
The RemoteFrameBuffer
class has a
method .get_stats()
that
returns a dict with performance metrics:
>>> w.reset_stats() # start measuring
... interact or run a test
>>> w.get_stats()
{
...
}
Performance tips¶
The framerate that can be obtained depends on a number of factors:
The size of a frame: smaller frames generally encode faster and result in smaller blobs, causing less strain on both CPU and IO.
How many widgets are drawing simultaneously: they use the same communication channel.
The
widget.quality
trait: lower quality results in faster encoding and smaller blobs.When using lossless images (
widget.quality == 100
), the entropy (information density) of a frame also matters, because for PNG, high entropy data takes longer to compress and results in larger blobs.
For more details about performance considerations in the implementation of jupyter_rfb
,
see issue #3.