protobuf protocol-buffers 序列化数据 gobs pickling string XML 用C实现的cPickle比pickle快1000倍 protobuf2 protobuf3 差异
场景:
浏览器请求--->python数据生成--->python-生成excel--->浏览器下载excel
目标:
重构为
浏览器请求--->python数据生成--->golang-生成excel--->浏览器下载excel
二阶目标:
后端全部golang实现
https://developers.google.com/protocol-buffers/
https://developers.google.com/protocol-buffers/docs/pythontutorial
Protocol Buffer Basics: Go | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/gotutorial
一阶子探索:
1、python --- protocol-buffers --- golang
Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data.
Protocol Buffers - Google's data interchange format
Protocol Buffer Basics: Go | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/gotutorial
Protocol Buffer Basics: Python | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/pythontutorial
序列化和获取结构数据的方式
How do you serialize and retrieve structured data like this? There are a few ways to solve this problem:
- Use gobs to serialize Go data structures. This is a good solution in a Go-specific environment, but it doesn't work well if you need to share data with applications written for other platforms.
- You can invent an ad-hoc way to encode the data items into a single string – such as encoding 4 ints as "12:3:-23:67". This is a simple and flexible approach, although it does require writing one-off encoding and parsing code, and the parsing imposes a small run-time cost. This works best for encoding very simple data.
- Serialize the data to XML. This approach can be very attractive since XML is (sort of) human readable and there are binding libraries for lots of languages. This can be a good choice if you want to share data with other applications/projects. However, XML is notoriously space intensive, and encoding/decoding it can impose a huge performance penalty on applications. Also, navigating an XML DOM tree is considerably more complicated than navigating simple fields in a class normally would be.
Protocol buffers are the flexible, efficient, automated solution to solve exactly this problem. With protocol buffers, you write a .proto description of the data structure you wish to store. From that, the protocol buffer compiler creates a class that implements automatic encoding and parsing of the protocol buffer data with an efficient binary format. The generated class provides getters and setters for the fields that make up a protocol buffer and takes care of the details of reading and writing the protocol buffer as a unit. Importantly, the protocol buffer format supports the idea of extending the format over time in such a way that the code can still read data encoded with the old format.
1/3、gobs -- golang pickling -- python
https://golang.org/pkg/encoding/gob/
Package gob manages streams of gobs - binary values exchanged between an Encoder (transmitter) and a Decoder (receiver). A typical use is transporting arguments and results of remote procedure calls (RPCs) such as those provided by package "net/rpc".
The implementation compiles a custom codec for each data type in the stream and is most efficient when a single Encoder is used to transmit a stream of values, amortizing the cost of compilation.
跨语言性差,局限在golang
Use Python pickling. This is the default approach since it's built into the language, but it doesn't deal well with schema evolution, and also doesn't work very well if you need to share data with applications written in C++ or Java.
11.1. pickle — Python object serialization — Python 2.7.16 documentation
https://docs.python.org/2/library/pickle.html
The pickle module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream is converted back into an object hierarchy. Pickling (and unpickling) is alternatively known as “serialization”, “marshalling,” [1] or “flattening”, however, to avoid confusion, the terms used here are “pickling” and “unpickling”.
This documentation describes both the pickle module and the cPickle module.
11.1. pickle — Python object serialization — Python 2.7.16 documentation
https://docs.python.org/2/library/pickle.html#module-cPickle
The cPickle module supports serialization and de-serialization of Python objects, providing an interface and functionality nearly identical to the pickle module. There are several differences, the most important being performance and subclassability.
First, cPickle can be up to 1000 times faster than pickle because the former is implemented in C. Second, in the cPickle module the callables Pickler() and Unpickler() are functions, not classes. This means that you cannot use them to derive custom pickling and unpickling subclasses. Most applications have no need for this functionality and should benefit from the greatly improved performance of the cPickle module.
The pickle data stream produced by pickle and cPickle are identical, so it is possible to use pickle and cPickle interchangeably with existing pickles. [10]
There are additional minor differences in API between cPickle and pickle, however for most applications, they are interchangeable. More documentation is provided in the pickle module documentation, which includes a list of the documented differences.
2/3、string
普通字符串的缺点是只能描述简单的数据结构
3/3、XML
跨语言性好,但是资源消耗高,性能差
更小更快
Developer Guide | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/overview
Developer Guide
Welcome to the developer documentation for protocol buffers – a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.
This documentation is aimed at Java, C++, or Python developers who want to use protocol buffers in their applications. This overview introduces protocol buffers and tells you what you need to do to get started – you can then go on to follow the tutorials or delve deeper into protocol buffer encoding. API reference documentation is also provided for all three languages, as well as language and style guides for writing .proto files.
What are protocol buffers?
Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format.
How do they work?
You specify how you want the information you're serializing to be structured by defining protocol buffer message types in .proto files. Each protocol buffer message is a small logical record of information, containing a series of name-value pairs. Here's a very basic example of a .proto file that defines a message containing information about a person:
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}
As you can see, the message format is simple – each message type has one or more uniquely numbered fields, and each field has a name and a value type, where value types can be numbers (integer or floating-point), booleans, strings, raw bytes, or even (as in the example above) other protocol buffer message types, allowing you to structure your data hierarchically. You can specify optional fields, required fields, and repeated fields. You can find more information about writing .proto files in the Protocol Buffer Language Guide.
Once you've defined your messages, you run the protocol buffer compiler for your application's language on your .proto file to generate data access classes. These provide simple accessors for each field (like name() and set_name()) as well as methods to serialize/parse the whole structure to/from raw bytes – so, for instance, if your chosen language is C++, running the compiler on the above example will generate a class called Person. You can then use this class in your application to populate, serialize, and retrieve Person protocol buffer messages. You might then write some code like this:
Person person;
person.set_name("John Doe");
person.set_id(1234);
person.set_email("jdoe@example.com");
fstream output("myfile", ios::out | ios::binary);
person.SerializeToOstream(&output);
Then, later on, you could read your message back in:
fstream input("myfile", ios::in | ios::binary);
Person person;
person.ParseFromIstream(&input);
cout << "Name: " << person.name() << endl;
cout << "E-mail: " << person.email() << endl;
You can add new fields to your message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. So if you have a communications protocol that uses protocol buffers as its data format, you can extend your protocol without having to worry about breaking existing code.
You'll find a complete reference for using generated protocol buffer code in the API Reference section, and you can find out more about how protocol buffer messages are encoded in Protocol Buffer Encoding.
Why not just use XML?
Protocol buffers have many advantages over XML for serializing structured data. Protocol buffers:
- are simpler
- are 3 to 10 times smaller
- are 20 to 100 times faster
- are less ambiguous
- generate data access classes that are easier to use programmatically
For example, let's say you want to model a person with a name and an email. In XML, you need to do:
<person>
<name>John Doe</name>
<email>jdoe@example.com</email>
</person>
while the corresponding protocol buffer message (in protocol buffer text format) is:
# Textual representation of a protocol buffer.
# This is *not* the binary format used on the wire.
person {
name: "John Doe"
email: "jdoe@example.com"
}
When this message is encoded to the protocol buffer binary format (the text format above is just a convenient human-readable representation for debugging and editing), it would probably be 28 bytes long and take around 100-200 nanoseconds to parse. The XML version is at least 69 bytes if you remove whitespace, and would take around 5,000-10,000 nanoseconds to parse.
Also, manipulating a protocol buffer is much easier:
cout << "Name: " << person.name() << endl;
cout << "E-mail: " << person.email() << endl;
Whereas with XML you would have to do something like:
cout << "Name: "
<< person.getElementsByTagName("name")->item(0)->innerText()
<< endl;
cout << "E-mail: "
<< person.getElementsByTagName("email")->item(0)->innerText()
<< endl;
However, protocol buffers are not always a better solution than XML – for instance, protocol buffers would not be a good way to model a text-based document with markup (e.g. HTML), since you cannot easily interleave structure with text. In addition, XML is human-readable and human-editable; protocol buffers, at least in their native format, are not. XML is also – to some extent – self-describing. A protocol buffer is only meaningful if you have the message definition (the .proto file).
编译器
binary format <--- the text format
https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.1
Protocol Buffers source code is hosted on GitHub: https://github.com/protocolbuffers/protobuf.
Our old Google Code repository is: https://code.google.com/p/protobuf/. We moved to GitHub on Aug 26, 2014 and no future changes will be made on the Google Code site. For latest code updates/issues, please visit our GitHub site.
Compiling Your Protocol Buffers
https://developers.google.com/protocol-buffers/docs/pythontutorial
protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/addressbook.proto
D:\pyPlusGlang\protoc-3.7.1-win64\bin>protoc -I=D:\pyPlusGlang\mysite\polls\ --python_out=D:\pyPlusG
lang\mysite\polls\ D:\pyPlusGlang\mysite\polls\addressbook.proto
D:\pyPlusGlang\mysite\polls\addressbook_pb2.py
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: addressbook.proto import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports) _sym_db = _symbol_database.Default() from google.protobuf import timestamp_pb2 as google_dot_protobuf_dot_timestamp__pb2 DESCRIPTOR = _descriptor.FileDescriptor(
name='addressbook.proto',
package='tutorial',
syntax='proto3',
serialized_options=_b('\n\024com.example.tutorialB\021AddressBookProtos\252\002$Google.Protobuf.Examples.AddressBook'),
serialized_pb=_b('\n\x11\x61\x64\x64ressbook.proto\x12\x08tutorial\x1a\x1fgoogle/protobuf/timestamp.proto\"\x87\x02\n\x06Person\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\x05\x12\r\n\x05\x65mail\x18\x03 \x01(\t\x12,\n\x06phones\x18\x04 \x03(\x0b\x32\x1c.tutorial.Person.PhoneNumber\x12\x30\n\x0clast_updated\x18\x05 \x01(\x0b\x32\x1a.google.protobuf.Timestamp\x1aG\n\x0bPhoneNumber\x12\x0e\n\x06number\x18\x01 \x01(\t\x12(\n\x04type\x18\x02 \x01(\x0e\x32\x1a.tutorial.Person.PhoneType\"+\n\tPhoneType\x12\n\n\x06MOBILE\x10\x00\x12\x08\n\x04HOME\x10\x01\x12\x08\n\x04WORK\x10\x02\"/\n\x0b\x41\x64\x64ressBook\x12 \n\x06people\x18\x01 \x03(\x0b\x32\x10.tutorial.PersonBP\n\x14\x63om.example.tutorialB\x11\x41\x64\x64ressBookProtos\xaa\x02$Google.Protobuf.Examples.AddressBookb\x06proto3')
,
dependencies=[google_dot_protobuf_dot_timestamp__pb2.DESCRIPTOR,]) _PERSON_PHONETYPE = _descriptor.EnumDescriptor(
name='PhoneType',
full_name='tutorial.Person.PhoneType',
filename=None,
file=DESCRIPTOR,
values=[
_descriptor.EnumValueDescriptor(
name='MOBILE', index=0, number=0,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='HOME', index=1, number=1,
serialized_options=None,
type=None),
_descriptor.EnumValueDescriptor(
name='WORK', index=2, number=2,
serialized_options=None,
type=None),
],
containing_type=None,
serialized_options=None,
serialized_start=285,
serialized_end=328,
)
_sym_db.RegisterEnumDescriptor(_PERSON_PHONETYPE) _PERSON_PHONENUMBER = _descriptor.Descriptor(
name='PhoneNumber',
full_name='tutorial.Person.PhoneNumber',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='number', full_name='tutorial.Person.PhoneNumber.number', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='type', full_name='tutorial.Person.PhoneNumber.type', index=1,
number=2, type=14, cpp_type=8, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=212,
serialized_end=283,
) _PERSON = _descriptor.Descriptor(
name='Person',
full_name='tutorial.Person',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='name', full_name='tutorial.Person.name', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='id', full_name='tutorial.Person.id', index=1,
number=2, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='email', full_name='tutorial.Person.email', index=2,
number=3, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='phones', full_name='tutorial.Person.phones', index=3,
number=4, type=11, cpp_type=10, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='last_updated', full_name='tutorial.Person.last_updated', index=4,
number=5, type=11, cpp_type=10, label=1,
has_default_value=False, default_value=None,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[_PERSON_PHONENUMBER, ],
enum_types=[
_PERSON_PHONETYPE,
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=65,
serialized_end=328,
) _ADDRESSBOOK = _descriptor.Descriptor(
name='AddressBook',
full_name='tutorial.AddressBook',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='people', full_name='tutorial.AddressBook.people', index=0,
number=1, type=11, cpp_type=10, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=330,
serialized_end=377,
) _PERSON_PHONENUMBER.fields_by_name['type'].enum_type = _PERSON_PHONETYPE
_PERSON_PHONENUMBER.containing_type = _PERSON
_PERSON.fields_by_name['phones'].message_type = _PERSON_PHONENUMBER
_PERSON.fields_by_name['last_updated'].message_type = google_dot_protobuf_dot_timestamp__pb2._TIMESTAMP
_PERSON_PHONETYPE.containing_type = _PERSON
_ADDRESSBOOK.fields_by_name['people'].message_type = _PERSON
DESCRIPTOR.message_types_by_name['Person'] = _PERSON
DESCRIPTOR.message_types_by_name['AddressBook'] = _ADDRESSBOOK
_sym_db.RegisterFileDescriptor(DESCRIPTOR) Person = _reflection.GeneratedProtocolMessageType('Person', (_message.Message,), dict( PhoneNumber = _reflection.GeneratedProtocolMessageType('PhoneNumber', (_message.Message,), dict(
DESCRIPTOR = _PERSON_PHONENUMBER,
__module__ = 'addressbook_pb2'
# @@protoc_insertion_point(class_scope:tutorial.Person.PhoneNumber)
))
,
DESCRIPTOR = _PERSON,
__module__ = 'addressbook_pb2'
# @@protoc_insertion_point(class_scope:tutorial.Person)
))
_sym_db.RegisterMessage(Person)
_sym_db.RegisterMessage(Person.PhoneNumber) AddressBook = _reflection.GeneratedProtocolMessageType('AddressBook', (_message.Message,), dict(
DESCRIPTOR = _ADDRESSBOOK,
__module__ = 'addressbook_pb2'
# @@protoc_insertion_point(class_scope:tutorial.AddressBook)
))
_sym_db.RegisterMessage(AddressBook) DESCRIPTOR._options = None
# @@protoc_insertion_point(module_scope)
https://developers.google.com/protocol-buffers/docs/pythontutorial
protobuf2 protobuf3 差异
addressbook.proto: Explicit default values are not allowed in proto3.
addressbook.proto: Required fields are not allowed in proto3.
addressbook.proto:8:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'o
ptional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
Language Guide | Protocol Buffers | Google Developers
https://developers.google.com/protocol-buffers/docs/proto
读写消息 protobuf二进制文件
D:\pyPlusGlang\LProtocol-buffers>python pb2ReadingAMessage.py myADDRESS_BOOK_FILE
Person ID: 1
Name: a001
Can't test non-submessage field "Person.email" for presence in proto3.
Mobile phone #:
152001 D:\pyPlusGlang\LProtocol-buffers>python pb2WritingAMessage.py myADDRESS_BOOK_FILE
['pb2WritingAMessage.py', 'myADDRESS_BOOK_FILE']
Enter person ID number: 002
Enter name: a002
Enter email address (blank for none): b@qq.cn
Enter a phone number (or leave blank to finish): 142002
Is this a mobile, home, or work phone? home
Enter a phone number (or leave blank to finish): D:\pyPlusGlang\LProtocol-buffers>python pb2ReadingAMessage.py myADDRESS_BOOK_FILE
Person ID: 1
Name: a001
Can't test non-submessage field "Person.email" for presence in proto3.
Mobile phone #:
152001
Person ID: 2
Name: a002
Can't test non-submessage field "Person.email" for presence in proto3.
Home phone #:
142002 D:\pyPlusGlang\LProtocol-buffers>
address_book = addressbook_pb2.AddressBook()
写protobuf文件
# Add an address.
PromptForAddress(address_book.people.add())
# Write the new address book back to disk.
f = open(fname, "wb")
f.write(address_book.SerializeToString())
读protobuf文件
# Read the existing address book.
f = open(fname, "rb")
address_book.ParseFromString(f.read())
一阶
python生成的protobuf文件被golang读取 反序列化
生成golang
golang/protobuf: Go support for Google's protocol buffers
https://github.com/golang/protobuf
- Grab the code from the repository and install the
protopackage. The simplest way is to rungo get -u github.com/golang/protobuf/protoc-gen-go. The compiler plugin,protoc-gen-go, will be installed in$GOPATH/binunless$GOBINis set. It must be in your$PATHfor the protocol compiler,protoc, to find it.
protoc -I=$SRC_DIR --go_out=$DST_DIR $SRC_DIR/addressbook.proto
protoc -I=D:\pyPlusGlang\LProtocol-buffers --go_out=D:\pyPlusGlang\LProtocol-buffers D:\pyPlusGlang\LProtocol-buffers\SearchRequest.proto
protoc.exe 需要protoc-gen-go.exe,会从当前目录开始查找
python
成功读写文件
D:\pyPlusGlang\LProtocol-buffers>python pb3ReadingAMessage.py sRqs_pb3
query: appleVSbanana
page_number: 1
result_per_page: 4 D:\pyPlusGlang\LProtocol-buffers>python pb3WritingAMessage.py sRqs_pb3
['pb3WritingAMessage.py', 'sRqs_pb3']
Enter query: aANDb
Enter page_number: 123
Enter result_per_page: 789 D:\pyPlusGlang\LProtocol-buffers>python pb3ReadingAMessage.py sRqs_pb3
query: appleVSbanana
page_number: 1
result_per_page: 4
query: aANDb
page_number: 123
result_per_page: 789 D:\pyPlusGlang\LProtocol-buffers>
syntax = "proto3";
message SearchRequest {
string query = 1;
int32 page_number = 2;
int32 result_per_page = 3;
}
message SearchRequestList {
repeated SearchRequest sR = 1;
}
D:\pyPlusGlang\LProtocol-buffers\SearchRequestList_pb2.py
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: SearchRequestList.proto import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports) _sym_db = _symbol_database.Default() DESCRIPTOR = _descriptor.FileDescriptor(
name='SearchRequestList.proto',
package='',
syntax='proto3',
serialized_options=None,
serialized_pb=_b('\n\x17SearchRequestList.proto\"L\n\rSearchRequest\x12\r\n\x05query\x18\x01 \x01(\t\x12\x13\n\x0bpage_number\x18\x02 \x01(\x05\x12\x17\n\x0fresult_per_page\x18\x03 \x01(\x05\"/\n\x11SearchRequestList\x12\x1a\n\x02sR\x18\x01 \x03(\x0b\x32\x0e.SearchRequestb\x06proto3')
) _SEARCHREQUEST = _descriptor.Descriptor(
name='SearchRequest',
full_name='SearchRequest',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='query', full_name='SearchRequest.query', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='page_number', full_name='SearchRequest.page_number', index=1,
number=2, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
_descriptor.FieldDescriptor(
name='result_per_page', full_name='SearchRequest.result_per_page', index=2,
number=3, type=5, cpp_type=1, label=1,
has_default_value=False, default_value=0,
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=27,
serialized_end=103,
) _SEARCHREQUESTLIST = _descriptor.Descriptor(
name='SearchRequestList',
full_name='SearchRequestList',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='sR', full_name='SearchRequestList.sR', index=0,
number=1, type=11, cpp_type=10, label=3,
has_default_value=False, default_value=[],
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None,
serialized_options=None, file=DESCRIPTOR),
],
extensions=[
],
nested_types=[],
enum_types=[
],
serialized_options=None,
is_extendable=False,
syntax='proto3',
extension_ranges=[],
oneofs=[
],
serialized_start=105,
serialized_end=152,
) _SEARCHREQUESTLIST.fields_by_name['sR'].message_type = _SEARCHREQUEST
DESCRIPTOR.message_types_by_name['SearchRequest'] = _SEARCHREQUEST
DESCRIPTOR.message_types_by_name['SearchRequestList'] = _SEARCHREQUESTLIST
_sym_db.RegisterFileDescriptor(DESCRIPTOR) SearchRequest = _reflection.GeneratedProtocolMessageType('SearchRequest', (_message.Message,), dict(
DESCRIPTOR = _SEARCHREQUEST,
__module__ = 'SearchRequestList_pb2'
# @@protoc_insertion_point(class_scope:SearchRequest)
))
_sym_db.RegisterMessage(SearchRequest) SearchRequestList = _reflection.GeneratedProtocolMessageType('SearchRequestList', (_message.Message,), dict(
DESCRIPTOR = _SEARCHREQUESTLIST,
__module__ = 'SearchRequestList_pb2'
# @@protoc_insertion_point(class_scope:SearchRequestList)
))
_sym_db.RegisterMessage(SearchRequestList) # @@protoc_insertion_point(module_scope)
D:\pyPlusGlang\LProtocol-buffers\pb3WritingAMessage.py
#! /usr/bin/python
import SearchRequestList_pb2
import sys def PromptForSearchRequest(searchRequest):
searchRequest.query = input("Enter query: ")
searchRequest.page_number = int(input("Enter page_number: "))
searchRequest.result_per_page = int(input("Enter result_per_page: "))
print(sys.argv)
if len(sys.argv) != 2:
print("here is :", __file__, sys._getframe().f_lineno) # @print@ "Usage:", sys.argv[0], "SearchRequest__FILE"
sys.exit(-1) searchRequestList = SearchRequestList_pb2.SearchRequestList() # Read the existing SearchRequest book.
try:
f = open(sys.argv[1], "rb")
searchRequestList.ParseFromString(f.read())
f.close()
except IOError:
print("here is :", __file__,
sys._getframe().f_lineno) # @print@ sys.argv[1] + ": Could not open file. Creating a new one." # Add an SearchRequest.
PromptForSearchRequest(searchRequestList.sR.add()) # Write the new SearchRequest book back to disk.
f = open(sys.argv[1], "wb")
f.write(searchRequestList.SerializeToString())
f.close()
D:\pyPlusGlang\LProtocol-buffers\pb3ReadingAMessage.py
#! /usr/bin/python import SearchRequestList_pb2
import sys # Iterates though all people in the AddressBook and prints info about them.
def ListSr(searchRequestList):
for sR in searchRequestList.sR:
print("query:", sR.query)
print("page_number:", sR.page_number)
print("result_per_page:", sR.result_per_page) # Main procedure: Reads the entire address book from a file and prints all
# the information inside.
if len(sys.argv) != 2:
print("Usage:", sys.argv[0], "searchRequestList")
sys.exit(-1) searchRequestList = SearchRequestList_pb2.SearchRequestList() # Read the existing address book.
f = open(sys.argv[1], "rb")
searchRequestList.ParseFromString(f.read())
f.close() ListSr(searchRequestList)
protobuf protocol-buffers 序列化数据 gobs pickling string XML 用C实现的cPickle比pickle快1000倍 protobuf2 protobuf3 差异的更多相关文章
- protobuf Protocol Buffers 简介 案例 MD
Markdown版本笔记 我的GitHub首页 我的博客 我的微信 我的邮箱 MyAndroidBlogs baiqiantao baiqiantao bqt20094 baiqiantao@sina ...
- Xml,Json,Hessian,Protocol Buffers序列化对比
简介 这篇博客主要对Xml,Json,Hessian,Protocol Buffers的序列化和反序列化性能进行对比,Xml和Json的基本概念就不说了. Hessian:Hessian是一个轻量级的 ...
- protobuf(Protocol Buffers)java初体验
因为项目须要所以简单的研究了下protobuf.我也是參照网上的博客,所以大部分内容我也就不反复造轮子了.首先protobuf介绍点击这里,使用介绍点击这里,使用demo看这里. 我个人的第一个样例也 ...
- Protocol Buffers序列化原理
1. 使用Varint编码数据,越小的数据,用少的字节编码.如小于128的用一个字节编码,大于128的用多个字节编码.同时,每个字节最高为1或者0表示是否为数字的一部分. 2. 由于负数的补码表示很大 ...
- Hadoop基础-Protocol Buffers串行化与反串行化
Hadoop基础-Protocol Buffers串行化与反串行化 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 我们之前学习过很多种序列化文件格式,比如python中的pickl ...
- ProtoBuf3语法指南(Protocol Buffers)_上
0.说明 ProtoBuf3语法指南, 又称为proto3, 是谷歌的Protocol Buffers第3个版本. 本文基于官方英文版本翻译, 加上了自己的理解少量修改, 一共分为上下两部分. 1.序 ...
- Netty中如何序列化数据
JDK提供了ObjectOutputStream和ObjectInputStream,用于通过网络对POJO的基本数据类型和图进行序列化和反序列化.该API并不复杂,而且可以被应用于任何实现了java ...
- Netty(五):Netty中如何序列化数据
JDK提供了ObjectOutputStream和ObjectInputStream,用于通过网络对POJO的基本数据类型和图进行序列化和反序列化.该API并不复杂,而且可以被应用于任何实现了java ...
- 使用纳米 Protocol buffers 作为序列化数据
使用纳米 Protocol buffers 作为序列化数据 Protocol Buffers 是 Google 公司开发的一种数据描述语言,类似于XML能够将结构化数据序列化. 但是它更小, 更快, ...
随机推荐
- c# 定义和调用索引器
- Linux命令——su 、su -、sudo
前言 大部分Linux发行版的默认账户是普通用户,而更改系统文件或者执行某些命令,需要root身份才能进行,这就需要从当前用户切换到root用户. 切换用户身份有两个命令 su [-] usernam ...
- 运维开发笔记整理-使用Django编写helloworld
运维开发笔记整理-使用Django编写helloworld 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.创建Django项目 1>.创建Django项目 djang ...
- 使用VisualGDB和OpenOCD调试STM32L0开发板
本教程主要介绍如何配置VisualGDB和OpenOCD来调试STM32L0开发板的固件,使微控制器进入睡眠模式. 我们将创建一个NUCLEO-L031K6开发板的基本工程,并介绍当CPU进入休眠模式 ...
- cmake下cmake_c_flags,add_definitions
cmake_c_flags用来设置编译选项 如 -g -wall(不展示警告) add_definitions 添加编译参数,比如add_definitions(-DDEBUG)将在gcc命令行添加D ...
- 1205 CSRF跨站请求与django中的auth模块使用
目录 今日内容 昨日回顾 基于配置文件的编程思想 importlib模块 简单代码实现 跨站请求伪造csrf 1. 钓鱼网站 如何实现 模拟该现象的产生 2. 解决问题 解决 {% csrf_toke ...
- 解决在macOS下安装了python却没有pip命令的问题【经验总结】
可以使用brew直接安装python,但是安装完成了之后没有pip命令. pip是常用的python包管理工具,类似于java的maven.第一反应brew install pip,却提示没这货. 可 ...
- Ubuntu增加swap交换空间的步骤
1.首先用命令free查看系统内 Swap 分区大小. free -m total used free shared buffers cached Mem: 2012 1960 51 0 748 95 ...
- matlab(2) Logistic Regression: 画出样本数据点plotData
画出data数据 data数据 34.62365962451697,78.0246928153624,030.28671076822607,43.89499752400101,035.84740876 ...
- .NET Core初览
.NET Core 初览的应用场景为游戏服务器开发.所以测试在侧重点上更强于IO和密集型计算 网络IO: NetCore:60%CPU QPS 56W C++:31%CPU QPS 58W C++:6 ...