← Back
Editing: managers.cpython-38.pyc
U e5d� � @ sB d ddddgZ ddlZddlZddlZddlZddlZddlZddlZddlmZ ddl m Z d d l m Z d d l mZmZmZ d d l mZ d d l mZ d dl mZ d dl mZ zd dl mZ dZW n ek r� dZY nX dd� Ze�eje� dd� dD �Zed ek �r.dd� Z eD ]Z e�e e � �qG dd� de �Z di fd d �Z d d � Z!G d d!� d!e"�Z#d"d#� Z$d$d%� Z%G d&d'� d'e �Z&G d(d)� d)e �Z'e j(e j)fe j*e j+fd*�Z,G d+d � d e �Z-G d,d-� d-e.�Z/G d.d� de �Z0d/d0� Z1i fd1d2�Z2dld3d4�Z3G d5d6� d6e �Z4G d7d8� d8e �Z5dmd9d:�Z6G d;d<� d<e0�Z7G d=d>� d>e0�Z8G d?d@� d@e8�Z9G dAdB� dBe0�Z:G dCdD� dDe0�Z;G dEdF� dFe0�Z<G dGdH� dHe0�Z=e2dIdJ�Z>G dKdL� dLe>�Z?e2dMdN�Z@dOdPie@_Ae2dQdR�ZBe2dSdT�ZCdUdUdUdPdPdV�eC_AG dWdS� dSeC�ZDG dXd� de-�ZEeE�dYejF� eE�dZejF� eE�d[ejGe:� eE�d\ejHe8� eE�d]ejIe8� eE�d^ejJe8� eE�d_ejKe8� eE�d`ejLe9� eE�daejMe;� eE�dbejNeD� eE�dcee?� eE�ddeOe@� eE�d8e5e=� eE�d:e6eB� eE�d6e4e<� eEjdPe7dde� eEjdUddf� e�r>G dgdh� dh�ZPG didj� dje&�ZQG dkd� de-�ZRdS )n� BaseManager� SyncManager� BaseProxy�Token�SharedMemoryManager� N)�getpid)� format_exc� )� connection)� reduction�get_spawning_popen� ProcessError)�pool)�process)�util)� get_context)� shared_memoryTFc C s t j | j| �� ffS �N)�array�typecode�tobytes)�a� r �0/usr/lib64/python3.8/multiprocessing/managers.py� reduce_array- s r c C s g | ]}t ti |�� ��qS r )�type�getattr��.0�namer r r � <listcomp>1 s r )�items�keys�valuesc C s t t | �ffS r )�list��objr r r �rebuild_as_list3 s r' c @ s4 e Zd ZdZdZdd� Zdd� Zdd� Zd d � Zd S ) r z3 Type to uniquely identify a shared object ��typeid�address�idc C s ||| | _ | _| _d S r r( )�selfr) r* r+ r r r �__init__B s zToken.__init__c C s | j | j| jfS r r( �r, r r r � __getstate__E s zToken.__getstate__c C s |\| _ | _| _d S r r( �r, �stater r r � __setstate__H s zToken.__setstate__c C s d| j j| j| j| jf S )Nz %s(typeid=%r, address=%r, id=%r))� __class__�__name__r) r* r+ r. r r r �__repr__K s �zToken.__repr__N) r4 � __module__� __qualname__�__doc__� __slots__r- r/ r2 r5 r r r r r < s r c C s8 | � ||||f� | �� \}}|dkr*|S t||��dS )zL Send a message to manager using connection `c` and return response �#RETURNN)�send�recv�convert_to_error)�cr+ � methodname�args�kwds�kind�resultr r r �dispatchS s rD c C sd | dkr |S | dkrRt |t�s4td�|| t|����| dkrHtd| �S t|�S ntd�| ��S d S )N�#ERROR)� #TRACEBACK�#UNSERIALIZABLEz.Result {0!r} (kind '{1}') type is {2}, not strrG zUnserializable message: %s z Unrecognized message type {!r})� isinstance�str� TypeError�formatr � RemoteError� ValueError)rB rC r r r r= ] s �� r= c @ s e Zd Zdd� ZdS )rL c C s dt | jd � d S )NzM --------------------------------------------------------------------------- r zK---------------------------------------------------------------------------)rI r@ r. r r r �__str__m s zRemoteError.__str__N)r4 r6 r7 rN r r r r rL l s rL c C s2 g }t | �D ] }t| |�}t|�r |�|� q |S )z4 Return a list of names of methods of `obj` )�dirr �callable�append)r& �tempr �funcr r r � all_methodst s rT c C s dd� t | �D �S )zP Return a list of names of methods of `obj` which do not start with '_' c S s g | ]}|d dkr|�qS )r �_r r r r r r � s z"public_methods.<locals>.<listcomp>)rT r% r r r �public_methods s rV c @ s� e Zd ZdZdddddddd d g Zd d � Zd d� Zdd� Zdd� Zdd� Z dd� Z dd� Z dd� Z e e e d�Z d d � Zd d � Zd d!� Zd"d#� Zd$d%� Zd&e_d'd(� Zd)d*� Zd+d,� Zd-d.� Zd/S )0�ServerzM Server class which runs in a process controlled by a manager object �shutdown�create�accept_connection� get_methods� debug_info�number_of_objects�dummy�incref�decrefc C sx t |t�s td�|t|����|| _t�|�| _t | \}}||dd�| _ | j j | _ ddi| _ i | _ i | _t�� | _d S )Nz&Authkey {0!r} is type {1!s}, not bytes� )r* Zbacklog�0�Nr )rH �bytesrJ rK r �registryr �AuthenticationString�authkey�listener_client�listenerr* � id_to_obj�id_to_refcount�id_to_local_proxy_obj� threading�Lock�mutex)r, re r* rg � serializer�Listener�Clientr r r r- � s �� zServer.__init__c C s� t �� | _| t�� _zVt j| jd�}d|_|�� z | j�� sL| j�d� q4W n ttfk rf Y nX W 5 tjtjkr�t � d� tjt_tj t_ t� d� X dS )z( Run the server forever zresetting stdout, stderrr )�targetTr N)rm �Event� stop_eventr �current_process�_manager_server�sys�stdout� __stdout__r �debug� __stderr__�stderr�exit�Thread�accepter�daemon�start�is_set�wait�KeyboardInterrupt� SystemExit)r, r� r r r � serve_forever� s zServer.serve_foreverc C sN z| j �� }W n tk r& Y q Y nX tj| j|fd�}d|_|�� q d S )N�rs r@ T)ri Zaccept�OSErrorrm r �handle_requestr� r� )r, r> �tr r r r� � s zServer.accepterc C sL d } }}zTt �|| j� t �|| j� |�� }|\}}}}|| jksTtd| ��t| |�}W n tk r~ dt � f} Y n>X z||f|�|�}W n tk r� dt � f} Y n X d|f} z|� | � W nt tk �r> } zTz|� dt � f� W n tk �r Y nX t � d| � t � d|� t � d| � W 5 d} ~ X Y nX |� � dS )z) Handle a new connection Nz%r unrecognizedrF r: zFailure to send message: %rz ... request was %r� ... exception was %r)r Zdeliver_challengerg Zanswer_challenger< �public�AssertionErrorr � Exceptionr r; r �info�close) r, r> �funcnamerC �request�ignorer@ rA rS �msg�er r r r� � s4 zServer.handle_requestc C s� t �dt�� j� |j}|j}| j}| j� � �s�zBd }}|� }|\}}} } z|| \}} } W n^ t k r� } z@z| j | \}} } W n& t k r� } z| �W 5 d}~X Y nX W 5 d} ~ X Y nX || kr�t d|t |�| f ��t||�}z|| | �}W n, tk �r" } z d|f}W 5 d}~X Y nPX | �o4| �|d�}|�rj| �|||�\}}t|| j|�}d||ff}nd|f}W n� t k �r� |dk�r�dt� f}nNz,| j| }|| |||f| �| �}d|f}W n tk �r� dt� f}Y nX Y nP tk �r t �dt�� j� t�d � Y n tk �r< dt� f}Y nX zDz ||� W n2 tk �r~ } z|d t� f� W 5 d}~X Y nX W q$ tk �r� } z@t �d t�� j� t �d |� t �d |� |�� t�d� W 5 d}~X Y q$X q$dS )zQ Handle requests from the proxies in a particular process/thread z$starting server thread to service %rNz+method %r of %r object is not in exposed=%rrE �#PROXYr: rF z$got EOF -- exiting thread serving %rr rG z exception in thread serving %rz ... message was %rr� r )r r{ rm �current_threadr r< r; rj ru r� �KeyErrorrl �AttributeErrorr r r� �getrY r r* r �fallback_mapping�EOFErrorrx r~ r� r� )r, �connr< r; rj r? r& r� �identr@ rA �exposed� gettypeid�keZ second_keZfunction�resr� r� r) ZridentZrexposed�tokenZ fallback_funcrC r r r � serve_client� s� � � ( �� ��� � $� zServer.serve_clientc C s |S r r �r, r� r� r& r r r �fallback_getvalue5 s zServer.fallback_getvaluec C s t |�S r �rI r� r r r � fallback_str8 s zServer.fallback_strc C s t |�S r )�reprr� r r r � fallback_repr; s zServer.fallback_repr)rN r5 � #GETVALUEc C s d S r r �r, r> r r r r^ D s z Server.dummyc C s� | j �t g }t| j�� �}|�� |D ]<}|dkr&|�d|| j| t| j| d �dd� f � q&d�|�W 5 Q R � S Q R X dS )zO Return some info --- useful to spot problems with refcounting rb z %s: refcount=%s %sr N�K � ) ro r$ rk r" �sortrQ rI rj �join)r, r> rC r" r� r r r r\ G s ��zServer.debug_infoc C s t | j�S )z* Number of shared objects )�lenrk r� r r r r] W s zServer.number_of_objectsc C sL z:zt�d� |�d� W n ddl}|�� Y nX W 5 | j �� X dS )z' Shutdown this process z!manager received shutdown message�r: Nr N)ru �setr r{ r; � traceback� print_exc)r, r> r� r r r rX ^ s zServer.shutdownc O s� t | �dkr| ^}}}} n�| s(td��n�d|krDtdt | �d ��|�d�}t | �dkr~| ^}}} ddl}|jd tdd � nFd |kr�tdt | �d ��|�d �}| ^}} ddl}|jd tdd � t| �} |j�� |j| \}}}} |dk�r|�st | �dk�r t d ��| d } n || |�} |dk�r2t | �}|dk �rlt |t ��s\td� |t|����t|�t|� }dt| � } t�d|| � | t|�|f|j| < | |jk�r�d|j| < W 5 Q R X |�|| � | t|�fS )z> Create a new shared object and return its id � z8descriptor 'create' of 'Server' object needs an argumentr) �7create expected at least 2 positional arguments, got %dr � r Nz2Passing 'typeid' as keyword argument is deprecated)� stacklevelr> z-Passing 'c' as keyword argument is deprecatedz4Without callable, must have one non-keyword argumentz,Method_to_typeid {0!r}: type {1!s}, not dictz%xz&%r callable returned object with id %r)r� rJ �pop�warnings�warn�DeprecationWarning�tuplero re rM rV rH �dictrK r r$ r+ r r{ r� rj rk r_ ) r@ rA r, r> r) r� rP r� �method_to_typeid� proxytyper& r� r r r rY k sp � � � �� � �� z Server.createz$($self, c, typeid, /, *args, **kwds)c C s t | j|j d �S )zL Return the methods of the shared object indicated by token r )r� rj r+ )r, r> r� r r r r[ � s zServer.get_methodsc C s" |t �� _|�d� | �|� dS )z= Spawn a new thread to serve this connection r� N)rm r� r r; r� )r, r> r r r r rZ � s zServer.accept_connectionc C s� | j �� z| j| d7 < W nh tk r� } zJ|| jkrrd| j|< | j| | j|< | j| \}}}t�d|� n|�W 5 d }~X Y nX W 5 Q R X d S )Nr z&Server re-enabled tracking & INCREF %r)ro rk r� rl rj r r{ )r, r> r� r� r& r� r� r r r r_ � s �z Server.increfc C s� || j kr$|| jkr$t�d|� d S | j�Z | j | dkrXtd�|| j| | j | ���| j | d8 < | j | dkr�| j |= W 5 Q R X || j kr�d| j|< t�d|� | j� | j|= W 5 Q R X d S )NzServer DECREF skipping %rr z+Id {0!s} ({1!r}) has refcount {2:n}, not 1+r )Nr Nzdisposing of obj with id %r)rk rl r r{ ro r� rK rj )r, r> r� r r r r` � s, � �� z Server.decrefN)r4 r6 r7 r8 r� r- r� r� r� r� r� r� r� r� r^ r\ r] rX rY �__text_signature__r[ rZ r_ r` r r r r rW � s< � "Q� =rW c @ s e Zd ZdgZdZdZdZdS )�State�valuer r r� N)r4 r6 r7 r9 �INITIAL�STARTED�SHUTDOWNr r r r r� � s r� )�pickleZ xmlrpclibc @ s� e Zd ZdZi ZeZd"dd�Zdd� Zdd � Z d#d d �Z e d$d d��Z dd� Z d%dd�Zdd� Zdd� Zdd� Zdd� Zedd � �Zed d � �Ze d&d d!��ZdS )'r z! Base class for managers Nr� c C s\ |d krt �� j}|| _t �|�| _t� | _tj| j_ || _ t | \| _ | _ |pTt� | _d S r )r rv rg �_addressrf �_authkeyr� �_stater� r� � _serializerrh Z _Listener�_Clientr �_ctx)r, r* rg rp Zctxr r r r- s zBaseManager.__init__c C sf | j jtjkrP| j jtjkr&td��n*| j jtjkr>td��ntd�| j j���t| j | j | j | j �S )zX Return server object with serve_forever() method and address attribute �Already started server�Manager has shut down�Unknown state {!r}) r� r� r� r� r� r r� rK rW � _registryr� r� r� r. r r r � get_server s � �zBaseManager.get_serverc C s8 t | j \}}|| j| jd�}t|dd� tj| j_dS )z> Connect manager object to the server process �rg Nr^ ) rh r� r� r� rD r� r� r� r� )r, rq rr r� r r r �connect s zBaseManager.connectr c C s4 | j jtjkrP| j jtjkr&td��n*| j jtjkr>td��ntd�| j j���|dk rht|�sht d��t j dd�\}}| j j t| �j| j| j| j| j|||fd�| _d �d d � | jjD ��}t| �jd | | j_| j�� |�� |�� | _|�� tj| j _t j | t| �j | j| j| j| j | j fd d�| _ dS )z@ Spawn a server process for this manager object r� r� r� Nz initializer must be a callableF)Zduplexr� �:c s s | ]}t |�V qd S r r� )r �ir r r � <genexpr>A s z$BaseManager.start.<locals>.<genexpr>�-r �r@ Z exitpriority)!r� r� r� r� r� r r� rK rP rJ r ZPiper� ZProcessr � _run_serverr� r� r� r� �_processr� Z _identityr4 r r� r� r< r �Finalize�_finalize_managerr� rX )r, � initializer�initargs�reader�writerr� r r r r� ( sH � �� ��zBaseManager.startc C s^ t � t jt j� |dk r ||� | �||||�}|�|j� |�� t�d|j� |� � dS )z@ Create a server, report its address and run it Nzmanager serving at %r) �signal�SIGINT�SIG_IGN�_Serverr; r* r� r r� r� ) �clsre r* rg rp r� r� r� �serverr r r r� S s zBaseManager._run_serverc O sd | j jtjkstd��| j| j| jd�}z t |dd|f| |�\}}W 5 |�� X t || j|�|fS )zP Create a new shared object; return the token and exposed tuple zserver not yet startedr� NrY ) r� r� r� r� r� r� r� r� r� rD r )r, r) r@ rA r� r+ r� r r r �_createj s zBaseManager._createc C s* | j dk r&| j �|� | j �� s&d| _ dS )zC Join the manager process (if it has been spawned) N)r� r� �is_alive�r, �timeoutr r r r� v s zBaseManager.joinc C s2 | j | j| jd�}zt|dd�W �S |�� X dS )zS Return some info about the servers shared objects and connections r� Nr\ �r� r� r� r� rD �r, r� r r r � _debug_info s zBaseManager._debug_infoc C s2 | j | j| jd�}zt|dd�W �S |�� X dS )z5 Return the number of shared objects r� Nr] r� r� r r r �_number_of_objects� s z BaseManager._number_of_objectsc C sj | j jtjkr| �� | j jtjkrf| j jtjkr<td��n*| j jtjkrTtd��ntd�| j j���| S )NzUnable to start serverr� r� ) r� r� r� r� r� r� r r� rK r. r r r � __enter__� s �zBaseManager.__enter__c C s | � � d S r )rX �r, �exc_typeZexc_valZexc_tbr r r �__exit__� s zBaseManager.__exit__c C s� | � � r�t�d� z,|||d�}zt|dd� W 5 |�� X W n tk rR Y nX | jdd� | � � r�t�d� t| d�r�t�d � | �� | jd d� | � � r�t�d � t j |_ z t j |= W n tk r� Y nX dS ) zQ Shutdown the manager process; will be registered as a finalizer z#sending shutdown message to managerr� NrX g �?)r� zmanager still alive� terminatez'trying to `terminate()` manager processg�������?z#manager still alive after terminate)r� r r� r� rD r� r� �hasattrr� r� r� r� r �_address_to_localr� )r r* rg r1 r� r� r r r r� � s. z BaseManager._finalize_managerc C s | j S r )r� r. r r r r* � s zBaseManager.addressTc s� d| j kr| j�� | _� dkr"t� |p0t� dd�}|p@t� dd�}|r�t|�� �D ]8\}}t|�tksrt d| ��t|�tksRt d| ��qR|||� f| j�< |r �fdd�} �| _ t | �| � dS )z9 Register a typeid with the manager type r� N� _exposed_�_method_to_typeid_z%r is not a stringc s` t �d�� | j�f|�|�\}}� || j| | j|d�}| j|j| jd�}t|d d|jf� |S )Nz)requesting creation of a shared %r object��managerrg r� r� r` ) r r{ r� r� r� r� r* rD r+ )r, r@ rA r� Zexp�proxyr� �r� r) r r rR � s �z"BaseManager.register.<locals>.temp) �__dict__r� �copy� AutoProxyr r$ r! r rI r� r4 �setattr) r� r) rP r� r� r� � create_method�keyr� rR r r� r �register� s* � � zBaseManager.register)NNr� N)Nr )Nr )N)NNNNT)r4 r6 r7 r8 r� rW r� r- r� r� r� � classmethodr� r� r� r� r� r� r� � staticmethodr� �propertyr* r r r r r r � s8 � + � �c @ s e Zd Zdd� Zdd� ZdS )�ProcessLocalSetc C s t �| dd� � d S )Nc S s | � � S r )�clearr% r r r �<lambda>� � z*ProcessLocalSet.__init__.<locals>.<lambda>)r �register_after_forkr. r r r r- � s zProcessLocalSet.__init__c C s t | �dfS rc )r r. r r r � __reduce__� s zProcessLocalSet.__reduce__N)r4 r6 r7 r- r r r r r r � s r c @ s� e Zd ZdZi Ze�� Zd dd�Zdd� Z d i fd d �Z d d � Z dd� Z e dd� �Zdd� Zdd� Zdd� Zdd� Zdd� ZdS ) r z. A base for proxies of shared objects NTFc C s� t j�8 t j�|jd �}|d kr:t�� t� f}|t j|j< W 5 Q R X |d | _|d | _ || _ | j j | _ || _ || _t| d | _|| _|d k r�t�|�| _n"| j d k r�| j j| _n t�� j| _|r�| �� t�| t j� d S )Nr r )r �_mutexr� r� r* r ZForkAwareLocalr �_tls�_idset�_tokenr+ �_id�_managerr� rh r� �_owned_by_managerr rf r� rv rg �_increfr � _after_fork) r, r� rp r� rg r� r_ � manager_ownedZ tls_idsetr r r r- s* zBaseProxy.__init__c C sd t �d� t�� j}t�� jdkr4|dt�� j 7 }| j| jj | j d�}t |d d|f� || j _ d S )Nz making connection to managerZ MainThread�|r� rZ )r r{ r rv r rm r� r� r r* r� rD r r )r, r r� r r r �_connect- s zBaseProxy._connectr c C s� z | j j}W n6 tk rB t�dt�� j� | �� | j j}Y nX |� | j |||f� |� � \}}|dkrp|S |dkr�|\}}| j j |j d } | jj|_| || j| j | j|d�} | j|j| jd�}t|dd|jf� | S t||��dS ) zV Try to call a method of the referent and return a copy of the result z#thread %r does not own a connectionr: r� ���r� r� Nr` )r r r� r r{ rm r� r r r; r r< r r� r) r r* r� r� r� rD r+ r= ) r, r? r@ rA r� rB rC r� r� r� r� r r r � _callmethod6 s6 � �zBaseProxy._callmethodc C s | � d�S )z9 Get a copy of the value of the referent r� �r r. r r r � _getvalueT s zBaseProxy._getvaluec C s� | j rt�d| jj� d S | j| jj| jd�}t|d d| j f� t�d| jj� | j � | j � | j oj| j j }tj| tj| j| j|| j| j | jfdd�| _d S )Nz%owned_by_manager skipped INCREF of %rr� r_ z INCREF %r� r� )r r r{ r r+ r� r* r� rD r r �addr r� r� r �_decrefr Z_close)r, r� r1 r r r r Z s$ ��zBaseProxy._increfc C s� |� | j� |d ks |jtjkr�z2t�d| j� || j|d�}t|d d| jf� W q� t k r� } zt�d|� W 5 d }~X Y q�X nt�d| j� |s�t |d�r�t�dt � � j � |j�� |`d S )Nz DECREF %rr� r` z... decref failed %sz%DECREF %r -- manager already shutdownr z-thread %r has no more proxies so closing conn)�discardr+ r� r� r� r r{ r* rD r� r� rm r� r r r� )r� rg r1 ZtlsZidsetr� r� r� r r r r! n s � zBaseProxy._decrefc C sH d | _ z | �� W n0 tk rB } zt�d| � W 5 d }~X Y nX d S )Nzincref failed: %s)r r r� r r� )r, r� r r r r � s zBaseProxy._after_forkc C s^ i }t � d k r| j|d< t| dd�rB| j|d< tt| j| j|ffS tt| �| j| j|ffS d S )Nrg �_isautoFr� ) r r� r r� � RebuildProxyr r r� r �r, rA r r r r � s ��zBaseProxy.__reduce__c C s | � � S r )r )r, Zmemor r r � __deepcopy__� s zBaseProxy.__deepcopy__c C s dt | �j| jjt| �f S )Nz <%s object, typeid %r at %#x>)r r4 r r) r+ r. r r r r5 � s �zBaseProxy.__repr__c C s: z | � d�W S tk r4 t| �dd� d Y S X dS )zV Return representation of the referent (or a fall-back if that fails) r5 Nr z; '__str__()' failed>)r r� r� r. r r r rN � s zBaseProxy.__str__)NNNTF)r4 r6 r7 r8 r� r ZForkAwareThreadLockr r- r r r r r r! r r r& r5 rN r r r r r � s( � ) c C s� t t�� dd�}|rT|j|jkrTt�d|� d|d< |j|jkrT|j|j |j|j< |� dd�opt t�� dd� }| ||fd|i|��S ) z5 Function used for unpickling proxy objects. rw Nz*Rebuild a proxy owned by manager, token=%rTr r_ Z _inheritingF) r r rv r* r r{ r+ rl rj r� )rS r� rp rA r� r_ r r r r$ � s � �r$ c C sp t |�}z|| |f W S tk r* Y nX i }|D ]}td||f |� q4t| tf|�}||_||| |f< |S )zB Return a proxy type whose methods are given by `exposed` zOdef %s(self, /, *args, **kwds): return self._callmethod(%r, args, kwds))r� r� �execr r r� )r r� �_cacheZdicZmeth� ProxyTyper r r � MakeProxyType� s �� r* c C s� t | d }|dkrB|| j|d�}zt|dd| f�}W 5 |�� X |dkrX|dk rX|j}|dkrjt�� j}td| j |�}|| ||||d�} d| _ | S )z* Return an auto-proxy for `token` r Nr� r[ z AutoProxy[%s])r� rg r_ T) rh r* r� rD r� r rv rg r* r) r# ) r� rp r� rg r� r_ r� r� r) r� r r r r � s �r c @ s e Zd Zdd� Zdd� ZdS )� Namespacec K s | j �|� d S r )r� �updater% r r r r- � s zNamespace.__init__c C sZ t | j�� �}g }|D ]$\}}|�d�s|�d||f � q|�� d| jjd�|�f S )NrU z%s=%rz%s(%s)z, ) r$ r� r! � startswithrQ r� r3 r4 r� )r, r! rR r r� r r r r5 � s zNamespace.__repr__N)r4 r6 r7 r- r5 r r r r r+ � s r+ c @ s8 e Zd Zd dd�Zdd� Zdd� Zdd � Zeee�Zd S ) �ValueTc C s || _ || _d S r )� _typecode�_value)r, r r� �lockr r r r- s zValue.__init__c C s | j S r �r0 r. r r r r� s z Value.getc C s || _ d S r r2 �r, r� r r r r� s z Value.setc C s dt | �j| j| jf S )Nz %s(%r, %r))r r4 r/ r0 r. r r r r5 s zValue.__repr__N)T) r4 r6 r7 r- r� r� r5 r r� r r r r r. s r. c C s t � | |�S r )r )r �sequencer1 r r r �Array s r5 c @ s8 e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd S ) � IteratorProxy)�__next__r; �throwr� c C s | S r r r. r r r �__iter__ s zIteratorProxy.__iter__c G s | � d|�S )Nr7 r �r, r@ r r r r7 s zIteratorProxy.__next__c G s | � d|�S )Nr; r r: r r r r; s zIteratorProxy.sendc G s | � d|�S )Nr8 r r: r r r r8 s zIteratorProxy.throwc G s | � d|�S )Nr� r r: r r r r� ! s zIteratorProxy.closeN) r4 r6 r7 r� r9 r7 r; r8 r� r r r r r6 s r6 c @ s2 e Zd ZdZd dd�Zdd� Zdd � Zd d � ZdS ) � AcquirerProxy)�acquire�releaseTNc C s" |d kr|fn||f}| � d|�S �Nr< r )r, Zblockingr� r@ r r r r< ' s zAcquirerProxy.acquirec C s | � d�S �Nr= r r. r r r r= * s zAcquirerProxy.releasec C s | � d�S r> r r. r r r r� , s zAcquirerProxy.__enter__c C s | � d�S r? r r� r r r r� . s zAcquirerProxy.__exit__)TN)r4 r6 r7 r� r< r= r� r� r r r r r; % s r; c @ s6 e Zd ZdZd dd�Zd dd�Zdd � Zdd d �ZdS )�ConditionProxy)r< r= r� �notify� notify_allNc C s | � d|f�S �Nr� r r� r r r r� 4 s zConditionProxy.waitr c C s | � d|f�S )NrA r )r, �nr r r rA 6 s zConditionProxy.notifyc C s | � d�S )NrB r r. r r r rB 8 s zConditionProxy.notify_allc C sd |� }|r|S |d k r$t �� | }nd }d }|s`|d k rN|t �� }|dkrNq`| �|� |� }q,|S )Nr )�time� monotonicr� )r, Z predicater� rC ZendtimeZwaittimer r r �wait_for: s zConditionProxy.wait_for)N)r )N)r4 r6 r7 r� r� rA rB rG r r r r r@ 2 s r@ c @ s2 e Zd ZdZdd� Zdd� Zdd� Zd d d �ZdS ) � EventProxy)r� r� r r� c C s | � d�S )Nr� r r. r r r r� O s zEventProxy.is_setc C s | � d�S �Nr� r r. r r r r� Q s zEventProxy.setc C s | � d�S )Nr r r. r r r r S s zEventProxy.clearNc C s | � d|f�S rC r r� r r r r� U s zEventProxy.wait)N)r4 r6 r7 r� r� r� r r� r r r r rH M s rH c @ sN e Zd ZdZddd�Zdd� Zdd� Zed d � �Zed d � �Z ed d� �Z dS )� BarrierProxy)�__getattribute__r� �abort�resetNc C s | � d|f�S rC r r� r r r r� [ s zBarrierProxy.waitc C s | � d�S )NrL r r. r r r rL ] s zBarrierProxy.abortc C s | � d�S )NrM r r. r r r rM _ s zBarrierProxy.resetc C s | � dd�S )NrK )�partiesr r. r r r rN a s zBarrierProxy.partiesc C s | � dd�S )NrK )� n_waitingr r. r r r rO d s zBarrierProxy.n_waitingc C s | � dd�S )NrK )�brokenr r. r r r rP g s zBarrierProxy.broken)N) r4 r6 r7 r� r� rL rM r rN rO rP r r r r rJ Y s rJ c @ s( e Zd ZdZdd� Zdd� Zdd� ZdS ) �NamespaceProxy)rK � __setattr__� __delattr__c C s0 |d dkrt �| |�S t �| d�}|d|f�S )Nr rU r rK )�objectrK �r, r � callmethodr r r � __getattr__n s zNamespaceProxy.__getattr__c C s4 |d dkrt �| ||�S t �| d�}|d||f�S )Nr rU r rR )rT rR rK )r, r r� rV r r r rR s s zNamespaceProxy.__setattr__c C s0 |d dkrt �| |�S t �| d�}|d|f�S )Nr rU r rS )rT rS rK rU r r r rS x s zNamespaceProxy.__delattr__N)r4 r6 r7 r� rW rR rS r r r r rQ l s rQ c @ s* e Zd ZdZdd� Zdd� Zeee�ZdS )� ValueProxy)r� r� c C s | � d�S )Nr� r r. r r r r� � s zValueProxy.getc C s | � d|f�S rI r r3 r r r r� � s zValueProxy.setN)r4 r6 r7 r� r� r� r r� r r r r rX s rX � BaseListProxy)�__add__� __contains__� __delitem__� __getitem__�__len__�__mul__� __reversed__�__rmul__� __setitem__rQ �count�extend�index�insertr� �remove�reverser� �__imul__c @ s e Zd Zdd� Zdd� ZdS )� ListProxyc C s | � d|f� | S )Nrd r r3 r r r �__iadd__� s zListProxy.__iadd__c C s | � d|f� | S )Nri r r3 r r r ri � s zListProxy.__imul__N)r4 r6 r7 rk ri r r r r rj � s rj � DictProxy)r[ r\ r] r9 r^ rb r r r� r! r" r� �popitem� setdefaultr, r# r9 �Iterator� ArrayProxy)r^ r] rb � PoolProxy) Zapply� apply_asyncr� �imap�imap_unorderedr� �map� map_async�starmap� starmap_asyncr� Z AsyncResult)rr rv rx rs rt c @ s e Zd Zdd� Zdd� ZdS )rq c C s | S r r r. r r r r� � s zPoolProxy.__enter__c C s | � � d S r )r� r� r r r r� � s zPoolProxy.__exit__N)r4 r6 r7 r� r� r r r r rq � s c @ s e Zd ZdZdS )r a( Subclass of `BaseManager` which supports a number of shared object types. The types registered are those intended for the synchronization of threads, plus `dict`, `list` and `Namespace`. The `multiprocessing.Manager()` function creates started instances of this class. N)r4 r6 r7 r8 r r r r r � s �QueueZ JoinableQueuert rn �RLock� Semaphore�BoundedSemaphore� Condition�Barrier�Poolr$ r� )r� r )r c @ sL e Zd ZdZg fdd�Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dS )�_SharedMemoryTrackerz+Manages one or more shared memory segments.c C s || _ || _d S r �Zshared_memory_context_name� segment_names)r, r r� r r r r- � s z _SharedMemoryTracker.__init__c C s( t �d|�dt� � �� | j�|� dS )z6Adds the supplied shared memory block name to tracker.zRegister segment � in pid N)r r{ r r� rQ �r, � segment_namer r r �register_segment� s z%_SharedMemoryTracker.register_segmentc C sB t �d|�dt� � �� | j�|� t�|�}|�� |�� dS )z�Calls unlink() on the shared memory block with the supplied name and removes it from the list of blocks being tracked.zDestroy segment r� N) r r{ r r� rg r � SharedMemoryr� �unlink)r, r� Zsegmentr r r �destroy_segment� s z$_SharedMemoryTracker.destroy_segmentc C s" | j dd� D ]}| �|� qdS )z<Calls destroy_segment() on all tracked shared memory blocks.N)r� r� r� r r r r� � s z_SharedMemoryTracker.unlinkc C s( t �d| jj� dt� � �� | �� d S )NzCall z .__del__ in )r r{ r3 r4 r r� r. r r r �__del__� s z _SharedMemoryTracker.__del__c C s | j | jfS r r� r. r r r r/ � s z!_SharedMemoryTracker.__getstate__c C s | j |� d S r )r- r0 r r r r2 s z!_SharedMemoryTracker.__setstate__N) r4 r6 r7 r8 r- r� r� r� r� r/ r2 r r r r r� � s r� c @ sR e Zd Zejdddg Zdd� Zdd� Zde_d d � Zd d � Z d d� Z dd� Z dS )�SharedMemoryServer� track_segment�release_segment� list_segmentsc O sZ t j| f|�|� | j}t|t�r,t�|�}td|� dt� � ��| _ t � dt� � �� d S )NZshm_rU z"SharedMemoryServer started by pid ) rW r- r* rH rd �os�fsdecoder� r �shared_memory_contextr r{ )r, r@ �kwargsr* r r r r- s �zSharedMemoryServer.__init__c O st t | �dkr| d }n4d|kr(|d }n"| s6td��ntdt | �d ��ttj| d d�rhtj|d < tj| |�S ) z�Create a new distributed-shared object (not backed by a shared memory block) and return its id to be used in a Proxy Object.r� r� r) zDdescriptor 'create' of 'SharedMemoryServer' object needs an argumentr� r r Z_shared_memory_proxyr� )r� rJ r� r, re r� rW rY )r@ r� Ztypeodr) r r r rY s � zSharedMemoryServer.createz&($self, c, typeid, /, *args, **kwargs)c C s | j �� t�| |�S )zACall unlink() on all tracked shared memory, terminate the Server.)r� r� rW rX r� r r r rX ) s zSharedMemoryServer.shutdownc C s | j �|� dS )z?Adds the supplied shared memory block name to Server's tracker.N)r� r� �r, r> r� r r r r� . s z SharedMemoryServer.track_segmentc C s | j �|� dS )z�Calls unlink() on the shared memory block with the supplied name and removes it from the tracker instance inside the Server.N)r� r� r� r r r r� 2 s z"SharedMemoryServer.release_segmentc C s | j jS )zbReturns a list of names of shared memory blocks that the Server is currently tracking.)r� r� r� r r r r� 7 s z SharedMemoryServer.list_segmentsN) r4 r6 r7 rW r� r- rY r� rX r� r� r� r r r r r� s � r� c @ s< e Zd ZdZeZdd� Zdd� Zdd� Zdd � Z d d � Z d S ) r a� Like SyncManager but uses SharedMemoryServer instead of Server. It provides methods for creating and returning SharedMemory instances and for creating a list-like object (ShareableList) backed by shared memory. It also provides methods that create and return Proxy Objects that support synchronization across processes (i.e. multi-process-safe locks and semaphores). c O sN t jdkr ddlm} |�� tj| f|�|� t�| j j � dt � � �� d S )N�posixr )�resource_trackerz created by pid ) r� r � r� Zensure_runningr r- r r{ r3 r4 r )r, r@ r� r� r r r r- I s z SharedMemoryManager.__init__c C s t �| jj� dt� � �� d S )Nz.__del__ by pid )r r{ r3 r4 r r. r r r r� U s zSharedMemoryManager.__del__c C sh | j jtjkrP| j jtjkr&td��n*| j jtjkr>td��ntd�| j j���| �| j | j | j | j �S )z@Better than monkeypatching for now; merge into Server ultimatelyz"Already started SharedMemoryServerz!SharedMemoryManager has shut downr� ) r� r� r� r� r� r r� rK r� r� r� r� r� r. r r r r� Y s � �z SharedMemoryManager.get_serverc C sx | j | j| jd��\}tjdd|d�}zt|dd|jf� W n. tk rh } z|�� |�W 5 d}~X Y nX W 5 Q R X |S )zoReturns a new SharedMemory instance with the specified size in bytes, to be tracked by the manager.r� NT)rY �sizer� ) r� r� r� r r� rD r � BaseExceptionr� )r, r� r� Zsmsr� r r r r� f s z SharedMemoryManager.SharedMemoryc C sv | j | j| jd��Z}t�|�}zt|dd|jjf� W n0 tk rf } z|j� � |�W 5 d}~X Y nX W 5 Q R X |S )z�Returns a new ShareableList instance populated with the values from the input sequence, to be tracked by the manager.r� Nr� ) r� r� r� r � ShareableListrD Zshmr r� r� )r, r4 r� Zslr� r r r r� r s z!SharedMemoryManager.ShareableListN) r4 r6 r7 r8 r� r� r- r� r� r� r� r r r r r = s )NNNT)T)S�__all__rx rm r� r ZqueuerE r� r r� r r� r �contextr r r r r r r r Z HAS_SHMEM� ImportErrorr r Z view_typesr$ r' Z view_typerT r rD r= r� rL rT rV rW r� rq rr Z XmlListenerZ XmlClientrh r r� r r r$ r* r r+ r. r5 r6 r; r@ rH rJ rQ rX rY rj rl r� rp Z BasePoolProxyrq r ry rt rn rz r{ r| r} r~ r r� r� r� r r r r r �<module> s� � c � w 4 � � � �%8
Save File
Cancel