← Back
Editing: decoder.pyo
� ���hc @ s� d Z d Z d d l Z d d l Z e j r6 e Z n d d l m Z d d l m Z d d l m Z d Z e Z e d Z e j Z d � Z d � Z e d( e � Z e d e � Z e d* e � Z e d e � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z e e j e � Z e e j e � Z e e j e � Z! e e j e � Z" e e j e e j# � Z$ e e j e e j# � Z% e e j& d � Z' e e j( d � Z) e e j& d � Z* e e j( d � Z+ e � Z, e � Z- e e j e e. � Z/ d � Z0 d � Z1 d � Z2 d � Z3 e j4 d e j5 � Z6 d � Z7 d � Z8 d � Z9 d � Z: d! � Z; d" � Z< d# � Z= d$ � Z> d% � Z? d& � Z@ e@ � ZA d S(+ s� Code for decoding protocol buffer primitives. This code is very similar to encoder.py -- read the docs for that module first. A "decoder" is a function with the signature: Decode(buffer, pos, end, message, field_dict) The arguments are: buffer: The string containing the encoded message. pos: The current position in the string. end: The position in the string where the current message ends. May be less than len(buffer) if we're reading a sub-message. message: The message object into which we're parsing. field_dict: message._fields (avoids a hashtable lookup). The decoder reads the field and stores it into field_dict, returning the new buffer position. A decoder for a repeated field may proactively decode all of the elements of that field, if they appear consecutively. Note that decoders may throw any of the following: IndexError: Indicates a truncated message. struct.error: Unpacking of a fixed-width field failed. message.DecodeError: Other errors. Decoders are expected to raise an exception if they are called with pos > end. This allows callers to be lax about bounds checking: it's fineto read past "end" as long as you are sure that someone else will notice and throw an exception later on. Something up the call stack is expected to catch IndexError and struct.error and convert them to message.DecodeError. Decoders are constructed using decoder constructors with the signature: MakeDecoder(field_number, is_repeated, is_packed, key, new_default) The arguments are: field_number: The field number of the field we want to decode. is_repeated: Is the field a repeated field? (bool) is_packed: Is the field a packed field? (bool) key: The key to use when looking up the field within field_dict. (This is actually the FieldDescriptor but nothing in this file should depend on that.) new_default: A function which takes a message object as a parameter and returns a new instance of the default value for this field. (This is called for repeated fields and sub-messages, when an instance does not already exist.) As with encoders, we define a decoder constructor for every type of field. Then, for every field of every message class we construct an actual decoder. That decoder goes into a dict indexed by tag, so when we decode a message we repeatedly read a tag, look up the corresponding decoder, and invoke it. s kenton@google.com (Kenton Varda)i����N( t encoder( t wire_format( t messageg �i c s � � f d � } | S( s� Return an encoder for a basic varint value (does not include tag). Decoded values will be bitwise-anded with the given mask before being returned, e.g. to limit them to 32 bits. The returned decoder does not take the usual "end" parameter -- the caller is expected to do bounds checking after the fact (often the caller can defer such checking until later). The decoder returns a (value, new_pos) pair. c s� d } d } x� t j | | � } | | d @| >O} | d 7} | d @sg | � M} � | � } | | f S| d 7} | d k r t d � � q q Wd S( Ni i i i� i i@ s$ Too many bytes when decoding varint.( t sixt indexbytest _DecodeError( t buffert post resultt shiftt b( t maskt result_type( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt DecodeVarintu s ( ( R R R ( ( R R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _VarintDecoderk s c s5 d | d >� d | >d � � � � f d � } | S( s0 Like _VarintDecoder() but decodes signed values.i c s� d } d } x� t j | | � } | | d @| >O} | d 7} | d @su | � M} | � A� } � | � } | | f S| d 7} | d k r t d � � q q Wd S( Ni i i i� i i@ s$ Too many bytes when decoding varint.( R R R ( R R R R R ( R R t signbit( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR � s ( ( t bitsR R ( ( R R R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _SignedVarintDecoder� s i i@ i c C sQ | } x$ t j | | � d @r, | d 7} q W| d 7} t j | | | !� | f S( s� Read a tag from the buffer, and return a (tag_bytes, new_pos) tuple. We return the raw bytes of the tag rather than decoding them. The raw bytes can then be used to look up the proper decoder. This effectively allows us to trade some work that would be done in pure-python (decoding a varint) for work that is done in C (searching for a byte string in a hash table). In a low-level language it would be much cheaper to decode the varint and use that, but not in Python. i� i ( R R t binary_type( R R t start( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt ReadTag� s c s � � f d � } | S( s� Return a constructor for a decoder for fields of a particular type. Args: wire_type: The field's wire type. decode_value: A function which decodes an individual value, e.g. _DecodeVarint() c s� | r( t � � � � � f d � } | S| rk t j | � � � t � � � � � � � � f d � } | S� � f d � } | Sd S( Nc s� | j � � } | d k r6 | j � � | � � } n � | | � \ } } | | 7} | | k rp t d � � n x2 | | k r� � | | � \ } } | j | � qs W| | k r� | d =t d � � n | S( Ns Truncated message.i����s Packed element was truncated.( t gett Nonet setdefaultR t append( R R t endR t field_dictt valuet endpointt element( t decode_valuet keyt local_DecodeVarintt new_default( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt DecodePackedField� s c s� | j � � } | d k r6 | j � � | � � } n xn � | | � \ } } | j | � | � } | | | !� k s� | | k r9 | | k r� t d � � n | Sq9 Wd S( Ns Truncated message.( R R R R R ( R R R R R R R t new_pos( R R R! t tag_bytest tag_len( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt DecodeRepeatedField� s c s? � | | � \ | � <} | | k r; | � =t d � � n | S( Ns Truncated message.( R ( R R R R R ( R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt DecodeField� s ( t _DecodeVarintR t TagBytest len( t field_numbert is_repeatedt is_packedR R! R" R& R' ( R t wire_type( R R R! R$ R% sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt SpecificDecoder� s ( ( R. R R/ ( ( R R. sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _SimpleDecoder� s /c s � � f d � } t | | � S( s� Like SimpleDecoder but additionally invokes modify_value on every value before storing it. Usually modify_value is ZigZagDecode. c s% � | | � \ } } � | � | f S( N( ( R R R R# ( R t modify_value( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt InnerDecode s ( R0 ( R. R R1 R2 ( ( R R1 sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _ModifiedDecoder� s c s: t j � � � t j � � � � f d � } t | | � S( s� Return a constructor for a decoder for a fixed-width field. Args: wire_type: The field's wire type. format: The format string to pass to struct.unpack(). c s. | � } � � | | | !� d } | | f S( Ni ( ( R R R# R ( t formatt local_unpackt value_size( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR2 s ( t structt calcsizet unpackR0 ( R. R4 R2 ( ( R4 R5 R6 sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _StructPackDecoder s c s( t j � � f d � } t t j | � S( s� Returns a decoder for a float field. This code works around a bug in struct.unpack for non-finite 32-bit floating-point values. c s� | d } | | | !} | d d !d k r� | d d !d k r� | d d !d k rZ t | f S| d d !d k rw t | f St | f S� d | � d } | | f S( Ni i s �i s �i s �s �s <f( t _NANt _NEG_INFt _POS_INF( R R R# t float_bytesR ( R5 ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR2 ) s & ( R7 R9 R0 R t WIRETYPE_FIXED32( R2 ( ( R5 sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _FloatDecoder s c s( t j � � f d � } t t j | � S( sk Returns a decoder for a double field. This code works around a bug in struct.unpack for not-a-number. c sw | d } | | | !} | d d !d k rZ | d d !d k rZ | d d !d k rZ t | f S� d | � d } | | f S( Ni i s �i s �i s �s <d( R; ( R R R# t double_bytesR ( R5 ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR2 K s ( R7 R9 R0 R t WIRETYPE_FIXED64( R2 ( ( R5 sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _DoubleDecoderC s c s� � j � | r4 t � � � � � � f d � } | S| rz t j � t j � � t � � � � � � � � f d � } | S� � � f d � } | Sd S( Nc sF | j � � } | d k r6 | j � � | � � } n � | | � \ } } | | 7} | | k rp t d � � n x� | | k r| } t | | � \ } } | � j k r� | j | � qs | j s� g | _ n t j � t j � } | j j | | | | !f � qs W| | k rB| � j k r)| d =n | j d =t d � � n | S( Ns Truncated message.i����s Packed element was truncated.( R R R R t _DecodeSignedVarint32t values_by_numberR t _unknown_fieldsR R) R t WIRETYPE_VARINT( R R R R R R R t value_start_posR R$ ( t enum_typeR+ R R R! ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR" e s0 c s� | j � � } | d k r6 | j � � | � � } n x� t | | � \ } } | � j k rm | j | � n2 | j s� g | _ n | j j � | | | !f � | � } | | | !� k s� | | k r9 | | k r� t d � � n | Sq9 Wd S( Ns Truncated message.( R R R RD RE R RF R ( R R R R R R R R# ( RI R R! R$ R% ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR& � s c s� | } t | | � \ } } | | k r6 t d � � n | � j k rR | | � <nG | j sg g | _ n t j � t j � } | j j | | | | !f � | S( Ns Truncated message.( RD R RE RF R R) R RG R ( R R R R R RH t enum_valueR$ ( RI R+ R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR' � s ( RI R( R R) R RG R* ( R+ R, R- R R! R" R& R' ( ( RI R+ R R R! R$ R% sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt EnumDecodera s s <Is <Qs <is <qc s� t � t j � � � f d � � | rj t j | t j � � t � � � � � � � � � f d � } | S� � � f d � } | Sd S( s% Returns a decoder for a string field.c sD y � | d � SWn, t k r? } d | � j f | _ � n Xd S( Ns utf-8s %s in field: %s( t UnicodeDecodeErrort full_namet reason( t byte_strt e( R t local_unicode( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _ConvertToUnicode� s c s� | j � � } | d k r6 | j � � | � � } n x� � | | � \ } } | | } | | k rs t d � � n | j � | | | !� � | � } | | | !� k s� | | k r9 | Sq9 Wd S( Ns Truncated string.( R R R R R ( R R R R R R t sizeR# ( RR R R R! R$ R% ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR& � s c sU � | | � \ } } | | } | | k r: t d � � n � | | | !� | � <| S( Ns Truncated string.( R ( R R R R R RS R# ( RR R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR' � s N( R( R t text_typeR R) R t WIRETYPE_LENGTH_DELIMITEDR* ( R+ R, R- R R! R& R' ( ( RR R R RQ R! R$ R% sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt StringDecoder� s c sf t � | rL t j | t j � � t � � � � � � � � f d � } | S� � f d � } | Sd S( s$ Returns a decoder for a bytes field.c s� | j � � } | d k r6 | j � � | � � } n x � | | � \ } } | | } | | k rs t d � � n | j | | | !� | � } | | | !� k s� | | k r9 | Sq9 Wd S( Ns Truncated string.( R R R R R ( R R R R R R RS R# ( R R R! R$ R% ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR& s c sO � | | � \ } } | | } | | k r: t d � � n | | | !| � <| S( Ns Truncated string.( R ( R R R R R RS R# ( R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR' s N( R( R R) R RU R* ( R+ R, R- R R! R& R' ( ( R R R! R$ R% sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt BytesDecoder� s c s� t j | t j � � t � � � | rj t j | t j � � t � � � � � � � � � f d � } | S� � � � f d � } | Sd S( s$ Returns a decoder for a group field.c s� | j � � } | d k r6 | j � � | � � } n x� | j � � } | d k ro | j � � | � � } n | j � j | | | � } | � } | | | !� k s� | | k r� t d � � n | � } | | | !� k s� | | k r9 | Sq9 Wd S( Ns Missing group end tag.( R R R t addt _InternalParseR ( R R R R R R R# ( t end_tag_bytest end_tag_lenR R! R$ R% ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR& , s c s� | j � � } | d k r6 | j � � | � � } n | j | | | � } | � } | | | !� k st | | k r� t d � � n | S( Ns Missing group end tag.( R R R RY R ( R R R R R R R# ( RZ R[ R R! ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR' A s N( R R) R t WIRETYPE_END_GROUPR* t WIRETYPE_START_GROUP( R+ R, R- R R! R& R' ( ( RZ R[ R R! R$ R% sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt GroupDecoder s c si t � | rL t j | t j � � t � � � � � � � � f d � } | S� � � f d � } | Sd S( s& Returns a decoder for a message field.c s� | j � � } | d k r6 | j � � | � � } n x� � | | � \ } } | | } | | k rs t d � � n | j � j | | | � | k r� t d � � n | � } | | | !� k s� | | k r9 | Sq9 Wd S( Ns Truncated message.s Unexpected end-group tag.( R R R R RX RY ( R R R R R R RS R# ( R R R! R$ R% ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR& Y s ! c s� | j � � } | d k r6 | j � � | � � } n � | | � \ } } | | } | | k rp t d � � n | j | | | � | k r� t d � � n | S( Ns Truncated message.s Unexpected end-group tag.( R R R R RY ( R R R R R R RS R# ( R R R! ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR' o s N( R( R R) R RU R* ( R+ R, R- R R! R& R' ( ( R R R! R$ R% sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt MessageDecoderO s c sp t j d t j � � t j d t j � � t j d t j � � t � t � t } � � � � � f d � } | S( s Returns a decoder for a MessageSet item. The parameter is the message Descriptor. The message set message looks like this: message MessageSet { repeated group Item = 1 { required int32 type_id = 2; required string message = 3; } } i i i c s� | } d } d } d } x� � | | � \ } } | � k rT � | | � \ } } q | � k r� � | | � \ } } | | } } q | � k r� Pq t | | | | � } | d k r t d � � q q W| | k r� t d � � n | d k r t d � � n | d k rt d � � n | j j | � } | d k r�| j | � } | d k rr| j | | j j � � } n | j | | | � | k r�t d � � q�n2 | j s�g | _ n | j j t | | | !f � | S( Ni����s Missing group end tag.s Truncated message.s MessageSet item missing type_id.s MessageSet item missing message.s Unexpected end-group tag.( t SkipFieldR t Extensionst _FindExtensionByNumberR R R t message_typet _concrete_classRY RF R t MESSAGE_SET_ITEM_TAG( R R R R R t message_set_item_startt type_idt message_startt message_endR$ RS t extensionR ( t item_end_tag_bytesR t local_ReadTagt message_tag_bytest type_id_tag_bytes( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt DecodeItem� sF ( R R) R RG RU R\ R R( R` ( t descriptort local_SkipFieldRo ( ( Rk R Rl Rm Rn sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt MessageSetItemDecoder� s 0c s^ | � t j | j t j � � t � � � t � | j � � � � � � � � f d � } | S( s" Returns a decoder for a map field.c s � j � } | j � � } | d k rB | j � � | � � } n x� � | | � \ } } | | } | | k r t d � � n | j � | j | | | � | k r� t d � � n � r� | | j j | j � n | j | | j <| � } | | | !� k s | | k rE | SqE Wd S( Ns Truncated message.s Unexpected end-group tag.( Rd R R R R t ClearRY R t MergeFromR ( R R R R R t submsgR RS R# ( t is_message_mapR R Rc R! R$ R% ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt DecodeMap� s$ ( R R) t numberR RU R* R( Rc ( t field_descriptorR! Rv Rw ( ( Rv R R Rc R! R$ R% sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt MapDecoder� s ! c C sU x) t | | | d !� d @r+ | d 7} q W| d 7} | | k rQ t d � � n | S( s/ Skip a varint value. Returns the new position.i i� s Truncated message.( t ordR ( R R R ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _SkipVarint� s c C s) | d 7} | | k r% t d � � n | S( s0 Skip a fixed64 value. Returns the new position.i s Truncated message.( R ( R R R ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _SkipFixed64 s c C s> t | | � \ } } | | 7} | | k r: t d � � n | S( s9 Skip a length-delimited value. Returns the new position.s Truncated message.( R( R ( R R R RS ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _SkipLengthDelimited s c C sK xD t | | � \ } } t | | | | � } | d k r= | S| } q Wd S( s* Skip sub-group. Returns the new position.i����N( R R` ( R R R R$ R# ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _SkipGroup s c C s d S( sF Skipping an END_GROUP tag returns -1 to tell the parent loop to break.i����( ( R R R ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _EndGroup$ s c C s) | d 7} | | k r% t d � � n | S( s0 Skip a fixed32 value. Returns the new position.i s Truncated message.( R ( R R R ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _SkipFixed32) s c C s t d � � d S( s; Skip function for unknown wire types. Raises an exception.s Tag had invalid wire type.N( R ( R R R ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _RaiseInvalidWireType1 s c s= t t t t t t t t g � t j � � � f d � } | S( s" Constructs the SkipField function.c s+ t | d d !� � @} � | | | | � S( s Skips a field with the specified tag. |pos| should point to the byte immediately after the tag. Returns: The new position (after the tag value), or -1 if the tag is an end-group tag (in which case the calling loop should break). i i ( R{ ( R R R R$ R. ( t WIRETYPE_TO_SKIPPERt wiretype_mask( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyR` F s ( R| R} R~ R R� R� R� R t TAG_TYPE_MASK( R` ( ( R� R� sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt _FieldSkipper6 s l l ���� l l �� (B t __doc__t __author__R7 R t PY3t intt longt google.protobuf.internalR R t google.protobufR R= R< R; t DecodeErrorR R R R( t _DecodeSignedVarintt _DecodeVarint32RD R R0 R3 R: R@ RC RK RG t Int32Decodert Int64Decodert UInt32Decodert UInt64Decodert ZigZagDecodet SInt32Decodert SInt64DecoderR? t Fixed32DecoderRB t Fixed64Decodert SFixed32Decodert SFixed64Decodert FloatDecodert DoubleDecodert boolt BoolDecoderRV RW R^ R_ R) R] Re Rr Rz R| R} R~ R R� R� R� R� R` ( ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/decoder.pyt <module>O sv ; # O . % / 4 J .
Save File
Cancel