Duplicating a `multiprocessing.Queue` across spawned processes


It seems like a seldom discussed topic on Google and StackOverflow, since most questions misinterpret how to use multiprocessing.Queue (which I may very well be doing), or just explain the issue of Queue non-picklability.

On Linux 5.10.15-zen2-1-zen:

import multiprocessing


class Example:
  def __init__(self):
    self.rx_queue = multiprocessing.Queue()

  def poll(self):
    print("received queue?", self.rx_queue.get())


if __name__ == "__main__":
  queue_to_duplicate = multiprocessing.Queue()

  ex = Example()
  ex_proc = multiprocessing.Process(target=ex.poll)
  ex_proc.start()

  ex.rx_queue.put(queue_to_duplicate)
  ex_proc.join()
  # RuntimeError: Queue objects should only be shared between processes through inheritance

I have the following sample code, my intention is to be able to pass a multiprocessing.Queue object, or duplicate the file descriptor, so that I can recreate it in the receiving process. The essence of why I'm trying to do this is so that one primary process can initiate a communication channel given another processes' Queue, and then that other process can establish its own Queue to complete the duplex.

I may very well be missing an obvious alternative, but I do understand that Linux supports the dup* system calls, and that a multiprocessing.Queue is composed of Connection objects which lie thinly over file descriptors, which I could duplicate theoretically, but as for how, I'm not sure.

---------------Answer---------------
import multiprocessing


class Example:
  def __init__(self):
    self.rx_queue = multiprocessing.Queue()

  def poll(self):
    queue = self.rx_queue.get()
    queue.put("echo")


if __name__ == "__main__":
  manager = multiprocessing.Manager()
  queue_to_duplicate = manager.Queue()

  ex = Example()
  ex_proc = multiprocessing.Process(target=ex.poll)
  ex_proc.start()

  ex.rx_queue.put(queue_to_duplicate)
  ex_proc.join()

  print(queue_to_duplicate.get())
  # ... "echo"

Oversight on my part from the documentation, using multiprocessing.Manager works completely as intended for this purpose.


Previous : mongoose push an object into a nested array after matching element
Next : How to access the correct `this` inside a callback