Communication Patterns
Wrapyfi supports the publisher-subscriber (PUB/SUB) pattern as well as the request-reply (REQ/REP) pattern. The PUB/SUB pattern assumes message arguments are passed from the publisher-calling script to the publishing method. The publisher executes the method and the subscriber (listener) merely triggers the method call, awaits the publisher to execute the method, and returns the publisher’s method returns. The REQ/REP pattern on the other hand assumes arguments from the client (requester) are sent to the server (responder or replier). Once the server receives the request, it passes the arguments to its own method, executes it, and replies to the client back with its method returns.
Warning
in REQ/REP, the requester transmits all arguments passed to the method as a dictionary encoded as a string. This is not ideal for predefined services, where the service expects a certain object/message type. A better approach would include the option to pass a single item of a certain value and type
Publishers and Listeners/Subscribers (PUB/SUB)
The publishers and listeners of the same message type should have identical constructor signatures. The current Wrapyfi version supports
4 universal message types for all middleware. The extended types such as ROSMessage
and ROS2Message
are exclusive to the provided middleware.
YARP:
Note
YARP publishers remain persistent. To disable persistence, pass the argument persistent=False
to the @MiddlewareCommunicator.register
decorator.
All messages are transmitted using the yarp
Python bindings
Image: Transmits and receives a
cv2
ornumpy
image using eitheryarp.BufferedPortImageRgb
oryarp.BufferedPortImageFloat
. When JPG conversion is specified, it uses ayarp.BufferedPortBottle
message carrying a JPEG encoded string insteadAudioChunk: Transmits and receives a
numpy
audio chunk with the sound properties usingyarp.Port
transportingyarp.Sound
NativeObject: Transmits and receives a
json
string supporting all native Python objects,numpy
arrays and other formats usingyarp.BufferedPortBottle
ROS:
Warning
ROS requires a custom message to handle audio. This message must be compiled first before using Wrapyfi with ROS Audio. Refer to these instructions for compiling Wrapyfi ROS services and messages.
All messages are transmitted using the rospy
Python bindings as topic messages
Image: Transmits and receives a
cv2
ornumpy
image usingsensor_messages.msg.Image
. When JPG conversion is specified, uses thesensor_messages.msg.CompressedImage
message insteadAudioChunk: Transmits and receives a
numpy
audio chunk usingwrapyfi_ros_interfaces.msg.ROSAudioMessage
NativeObject: Transmits and receives a
json
string supporting all native Python objects,numpy
arrays, and other formats usingstd_msgs.msg.String
Properties: Transmits and receives parameters to/from the parameter server using the methods
rospy.set_param
androspy.get_param
respectivelyROSMessage: Transmits and receives a single ROS message per return decorator. Note that currently, only common ROS interface messages are supported and detected automatically. This means that messages defined in common interfaces such as std_msgs, geometry_msgs, and sensor_msgs can be directly returned by the method do not need to be converted to native types
ROS 2:
Warning
ROS 2 requires a custom message to handle audio. This message must be compiled first before using Wrapyfi with ROS 2 Audio. Refer to these instructions for compiling Wrapyfi ROS 2 services and messages.
All messages are transmitted using the rclpy
Python bindings as topic messages
Image: Transmits and receives a
cv2
ornumpy
image usingsensor_messages.msg.Image
. When JPG conversion is specified, uses thesensor_messages.msg.CompressedImage
message insteadAudioChunk: Transmits and receives a
numpy
audio chunk usingwrapyfi_ros2_interfaces.msg.ROS2AudioMessage
NativeObject: Transmits and receives a
json
string supporting all native Python objects,numpy
arrays, and other formats usingstd_msgs.msg.String
ROS2Message: Transmits and receives a single ROS 2 message per return decorator
ZeroMQ:
Note
ZeroMQ exchanges in REQ/REP rely on a broker with a dedicated socket. By default, Wrapyfi will not spawn a new connection to the socket when multiple threads are created. For multi-threaded applications, this leads to race conditions. We avoid that by detecting whether a new instance of the socket is available in the thread’s local storage. This multi-threading-friendly mode is enabled by passing multi_threaded=True
to the @MiddlewareCommunicator.register
decorator. This is only recommended when registering methods that are going to be multi-threaded.
All messages are transmitted using the zmq
Python bindings. Transmission follows the proxied XPUB/XSUB pattern
Image: Transmits and receives a
cv2
ornumpy
image wrapped in theNativeObject
construct. Note that allImage
types are transmitted as multipart messages, where the first element is the topic name and the second element is the header (e.g., timestamp), and the third element is the image itselfAudioChunk: Transmits and receives a
numpy
audio chunk wrapped in theNativeObject
constructNativeObject: Transmits and receives a
json
string supporting all native Python objects,numpy
arrays and other formats usingzmq context.socket(zmq.PUB).send_multipart
for publishing andzmq context.socket(zmq.SUB).receive_multipart
for receiving messages. Thezmq.PUB
socket is wrapped in azmq.proxy
to allow multiple subscribers to the same publisher. Note that allNativeObject
types are transmitted as multipart messages, where the first element is the topic name and the second element is the message itself (Except forImage
)
Servers and Clients (REQ/REP)
The servers and clients of the same message type should have identical constructor signatures. The current Wrapyfi version supports
3 universal message types for all middleware. The extended types such as ROSMessage
and ROS2Message
are exclusive to the provided middleware.
YARP:
All messages are transmitted using the yarp
Python bindings for RPC communication.
The requester encodes its arguments as a json
string supporting all native Python objects, numpy
arrays, and other formats using yarp.Bottle
.
The requester formats its arguments as ([args], {kwargs})
Image: Transmits and receives a
cv2
ornumpy
image encoded as ajson
string usingyarp.Bottle
. JPG conversion is currently not supportedAudioChunk: Transmits and receives a
numpy
audio chunk encoded as ajson
string usingyarp.Bottle
NativeObject: Transmits and receives a
json
string supporting all native Python objects,numpy
arrays, and other formats usingyarp.Bottle
ROS:
Warning
ROS requires a custom service to handle audio. This service must be compiled first before using Wrapyfi with ROS Audio. Refer to these instructions for compiling Wrapyfi ROS services and messages.
All messages are transmitted using the rospy
Python bindings as services.
The requester encodes its arguments as a json
string supporting all native Python objects, numpy
arrays, and other formats using std_msgs.msg.String
.
The requester formats its arguments as ([args], {kwargs})
Image: Transmits and receives a
cv2
ornumpy
image usingsensor_messages.msg.Image
JPG conversion is currently not supportedAudioChunk: Transmits and receives a
numpy
audio chunk usingwrapyfi_ros_interfaces.msg.ROSAudioMessage
NativeObject: Transmits and receives a
json
string supporting all native Python objects,numpy
arrays, and other formats usingstd_msgs.msg.String
ROS 2:
Warning
ROS 2 requires custom services to handle arbitrary messages. These services must be compiled first before using Wrapyfi in this mode. Refer to these instructions for compiling Wrapyfi ROS 2 services.
All messages are transmitted using the rclpy Python bindings as services.
The requester encodes its arguments as a json
string supporting all native Python objects, numpy
arrays, and other formats using std_msgs.msg.String
.
The requester formats its arguments as ([args], {kwargs})
Image: Transmits and receives a
cv2
ornumpy
image usingsensor_messages.msg.Image
AudioChunk: Transmits and receives a
numpy
audio chunk usingwrapyfi_ros2_interfaces.msg.ROS2AudioMessage
NativeObject: Transmits and receives a
json
string supporting all native Python objects,numpy
arrays, and other formats usingstd_msgs.msg.String
ZeroMQ:
All messages are transmitted using the zmq
Python bindings. Transmission follows the proxied XREP/XREQ pattern
The requester encodes its arguments as a json
string supporting all native Python objects, numpy
arrays, and other formats using zmq context.socket(zmq.REQ).send_multipart
.
The requester formats its arguments as ([args], {kwargs})
Image: Transmits and receives a
cv2
ornumpy
image wrapped in theNativeObject
constructAudioChunk: Transmits and receives a
numpy
audio chunk wrapped in theNativeObject
constructNativeObject: Transmits and receives a
json
string supporting all native Python objects,numpy
arrays, and other formats usingzmq context.socket(zmq.REP)
for replying andzmq context.socket(zmq.REQ)
for receiving messages
Publisher- and Listener-specific Arguments
Warning
Differences are expected between the returns of publishers and listeners, sometimes due to compression methods
(e.g., setting jpg=True
when transmitting an Image compresses the image but the encoding remains the same),
intentional setting of different devices for different tensors (refer to device mapping for tensors),
and differences in library versions between receiving and transmitting plugins (refer to plugins).
To direct arguments specifically toward the publisher or subscriber without exposing one or the other to the same argument values, the corresponding arguments can be added to the dictionary listener_kwargs
to control the listener only, or publisher_kwargs
to control the publisher only. Both dictionaries can be passed directly to the Wrapyfi decorator.
Since the transmitting and receiving arguments should generally be the same regardless of the communication pattern, publisher_kwargs
and listener_kwargs
also apply to the servers and clients respectively.