Protoscope is a simple, human-editable language for representing and emitting the Protobuf wire format.

  • By Protocol Buffers
  • Last update: Dec 29, 2022
  • Comments: 4

Protoscope

Protobuf + Rotoscope

Protoscope is a simple, human-editable language for representing and emitting the Protobuf wire format. It is inspired by, and is significantly based on, DER ASCII, a similar tool for working with DER and BER, wire formats of ASN.1.

Unlike most Protobuf tools, it is completely ignorant of schemata specified in .proto files; it has just enough knowledge of the wire format to provide primitives for constructing messages (such as field tags, varints, and length prefixes). A disassembler is included that uses heuristics to try convert encoded Protobuf into Protoscope, although the heuristics are necessarily imperfect.

We provide the Go package github.com/protocolbuffers/protoscope, as well as the protoscope tool, which can be installed with the Go tool via

go install github.com/protocolbuffers/protoscope/cmd/[email protected]

These tools may be used to create test inputs by taking an existing proto, dissassembling with protoscope, making edits, and then reassembling with protoscope -s. This avoids having to manually fix up all the length prefixes. They may also be used to inspect proto files (or things that look like them.)

For the language specification and basic examples, see language.txt. Example disassembly can be found under ./testdata.

Backwards compatibility

The Protoscope language itself may be extended over time, but the intention is for extensions to be backwards-compatible. Specifically:

  • The command-line interface to protoscope will remain compatible, though new options may be added in the future.

  • Previously valid Protoscope will remain valid and produce the same output. In particular, checking in test data as Protoscope text should be future-proof.

  • Previously invalid Protoscope may become valid in the future if the language is extended.

  • Disassembly is necessarily a heuristic, so its output may change over time, but it is guaranteed to produce Protoscope output that will reassemble to the original byte string. protoscope | protoscope -s is always equivalent to cat.

Disclaimer

This is not an official Google project.

Download

protoscope.zip

Comments(4)

  • 1

    Implement schema support for disassembly

    This PR adds a number of new features to the disassembler, foremost the ability to consume a message descriptor that makes the disassembly heuristics much smarter. Changes include:

    1. Given a descriptor, correct disassembly of: a. ZigZag values. c. Unsigned values. b. Bools. c. Subnormal floats. d. Packed fields.
    2. Given a descriptor, remarks with field names and enum value names.
    3. Smarter collapse of braces during printing.
    4. Better alignment of disassembler remarks.
    5. Flags for showing the spec (showing it in -help is rude) and whether to show bitpatterns of floats (off by default now).
  • 2

    Add more output flags

    This change does some cleanups that allow us to add three new flags for controlling the dumper:

    • -explicit-length-prefixes, which emits a literal integer instead of a { for type 2 records.
    • -no-groups, which disables emitting !{}.
    • -explicit-wire-types, which disables outputing fields without an explicit wire type.
  • 3

    Implement a syntax for groups

    This change adds the following syntax for specifying groups:

    1: !{
  # Stuff goes here.
}

    This will emit matching pairs of 1:SGROUP and 1:EGROUP tags where appropriate. The disassembler will now try to emit !{} where possible instead of SGROUP and EGROUP, but invalid messages may need to fall back on the old syntax.

  • 4

    Hello, how can I restore.proto， I hope you can help me， thanks

    hexdata.txt 08071093F4FF8E8FEE8DA36F18B84520082A150A0D636F6D70726573735F747970651204677A69702AA7020A0F696D2D696E7465726E616C5F657874129302696E7465726E616C5F7372633A707573687365727665727C6E6578745F637572736F723A742D313636383334363333353134395F722D373136353439323934323933313531383734395F642D315F752D315F682D315F6368642D5F6368752D5F7264632D317C7773735F707573685F726F6F6D5F69643A373136353434353939353034323534303332317C7773735F707573685F6469643A373131373233383836393531393137333136317C7773735F707573685F6C6F675F69643A383031383135373134343831333539393235317C7773735F66657463685F6D733A313636383334363333353036337C7773735F707573685F6D733A313636383334363333353134397C7773735F6D73675F747970653A722A4E0A09696D2D637572736F721241742D313636383334363333353134395F722D373136353439323934323933313531383734395F642D315F752D315F682D315F6368642D5F6368752D5F7264632D312A170A06696D2D6E6F77120D313636383334363333353134392A190A0E696D2D6C6976655F637572736F721207642D315F752D31320270623A036D736742CD0C1F8B08000000000000FFEC996D6C1C4719C7BD7BE7F83238C659DE8EF081C39110059D3DB3EF7B122F7612C7717C8D733E276E041DCDEDCEDDADEF6EF7B2BB77675F52845028980071F9108908439A8AA60A82A68E2AA586D22A6D0444F005099080482DCAE5DC22BE54CD4724743EC779BBA6711C43D5EE7E399D66E679669EF9CF3EBF7D06FC43011FDD4F533A71BD382DA4A813A7AE4B32947B41018B5CEBB6EEEF1D3F72F16FBFFAF5793DFCE4F1C77FF35CEDB9F33A64067EC9818FE5CD32C58E6D1730B53CEA60CF26AEC77DF2108C955CEA3C16B93AF7B32B0B8F1F4231D7734C2BF3D8962EB0716B4A4D2F3D91D940CF1B9B431FE23A41C75675DB0E657030321B38C3FC7933F8D3E650EDE8E5B77FC0761FBB367B32B0A53DD77822CCF06150CD7A5ED18DF5F51511EA35ECD2B469154DBD57B70B7DA4420BB40F413885206CFE8B9232F188D3E7D96E54B7A2A4EC4521441246BC64488452216D286949D2D322510824484784F01A85BD93459AF972DAB10B5F1421829280A0A09E61AA60BCE1BEE15D8E569AB1BA6D126621D3B7DCD2D70802CE38C4A0384FCB348FCB1246B0B76865BEEE15F3E5A89D9AEC350B2443C3DD910864063E15063D9F58FCFE538B6766EABF98A92F9C46B0BE70BA76F468FDE727E7998ED02CD3FDD930B3C0BCCA407082F9DABA4EE61483C1FEEBF6857BB0BF146F5C24D3CB4EF8298C2046622B0F9D914EC85C606A00FCA51D6C5C7C76AE76F689FA8923DDED61B6E7C57670890DB1DC45167C7535FED3C472F57C29757D7532CE9B99AC67D8150B4B2D2671C3BAB6AED6EF656F566B9D7BE89E47827FB2A1207799058F3EB8E56AAD16FCE8835BF0DDEDAF5D0CADEC735F58C5D8C406106C6BFB127760F997BBA161F8D4C2B91FBFCC7233ED21863BD20E3AEA0BA7CDABF33FEA6E0F333DFF0A82530D6DCFB120B1B6ED68A9B9C4DAB6E0EE36EF2FEC2DD5FBE977E90F161A9A7D9E05C9B506A9A592926B0DD3DDADDE6FA05AAAF233EF3AE20E2D5E571CBCFCFA6B975FB9F674C737D9B67976A3AAF0B2C60BB2F216ABC4C7C4CAC864FF407F7F7FFFAE8989BDA2B22F33B237313436EE68DB54C518DA35F1C823C9B896A866FB0B78F71E7938399631ECB381C09B81FF630E7C3340C1DED53827969EB51DDC881DF6480697F9168EE1D681AE9ECEFAFCCC95F3734D3FB341E654B0C91C97823B9BC17A58D5BD6482227B64FA40BC42C8CEA11D3495A6DE5425572D233B3E319E4DC2DDD0328431A3343831902A5813767A746CB0E059BBB769F9DC1033CA4CCCFE8EFD0E13E07CBEF940F38D9F92D73F25FB10F9002172157CE3A7EEFB4FDD3E1CDE031CFA7CF3DEE49BF0EFD951E687CCBF39F0F1E5E9DD5219328BDC432B75A137BE7162F1A56F470EA158D1A1E9953A11EFD7897C8EF239CAE7289FA37C14F0EB447E9DE883C8513D9F0B75715D37A38ECA5C603E023616A671DACEE7ED0AB7A1F6AD7357CF3DD3C3869867984078E57E0EBCBAE91DAEF79EDC045E0BBEC3F5DEA9E36FCDFFFD8EEBBDA7830FE07AEFF5C09DD8F6C700B81408D59F78F90FDFBD7691E9FEE9D9B75FDAB4A5B376EC4CEDD917AEBC78B236772CC20C57C1D45AF8CD8C421509584E0B342DA78921688A487455E591206B905745CD40446D4D6F0D807A9EE9DED400A8B60B0C00A19E0D20186AE3DA96CE4757356B1A25D332B325626AEA6D87C4AC56656747B9B82F3B8A73C2A83838BCDD8BE71E1E15F68C1CEC9FDCEE0D0D25154293467C687CE990FC0F757A4B8857E4BAA7E8C8B99D0707C64B245920C37B93B9E1EC417597551DD99FEEDF95B2C5C9B2E51DB0B6A752396744DB7730571A3C80C50445037A263A98C1E2CD654D7F5FDF3FFBBAFC39570BAEC7E79CFF5E78DFEB67F5696C250F71635E14C9B22A88B220485016B01355902C891AAF89BC262009A98AA861238A70298A70B6D92A8BB2A4421949A22C4111EB59238AF56C298A1D438FA2F06247E4AF3F7965E6B7F0F3FF61CD86882D92C7AEA3C78A2537EB52A74C9DC3169DF2B05E725CDB89ADC31C0E575C1737DC358F9269C496068992A64950E425110AFC4D9D8CA50E48819A067959B9D190B7338DC12A442A9214248A2A12244D51C5669F34F5F42C2EB8B15B16706378CB96829BC1DE7491C69CC315C77071AEECC696712161DB8504B172CBC0B01217A84189D74409A908DFD475CC239E7B7B5F81474841488248F8CA878798FF020000FFFF010000FFFFE0A2C71089240000 "xxd -r -ps hexdata.txt | protoscope" 1: 7 2: 8018157144813599251 3: 8888 4: 8 5: { 1: {"compress_type"} 2: {"gzip"} } 5: { 1: {"im-internal_ext"} 2: { "internal_src:pushserver|next_cursor:t-1668346335149_r-7165492942931518749_d-1_u-" "1_h-1_chd-_chu-_rdc-1|wss_push_room_id:7165445995042540321|wss_push_did:71172388" "69519173161|wss_push_log_id:8018157144813599251|wss_fetch_ms:1668346335063|wss_p" "ush_ms:1668346335149|wss_msg_type:r" } } 5: { 1: {13: 1.677723921655375e243} # 0x726f737275632d6di64 2: {"t-1668346335149_r-7165492942931518749_d-1_u-1_h-1_chd-_chu-_rdc-1"} } 5: { 1: {"im-now"} 2: {"1668346335149"} } 5: { 1: { 13: 4.7395440062453295e170 # 0x635f6576696c2d6di64 14: 4.742815e30i32 # 0x726f7372i32 } 2: {"d-1_u-1"} } 6: {14: 98} 7: {"msg"} 8: {1f8b08000000000000ffec996d6c1c4719c7bd7be7f83238c659de8ef081c39110059d3db3ef7b122f7612c7717c8d733e276e041dcdedceddadef6ef7b2bb77675f52845028980071f9108908439a8aa60a82a68e2aa586d22a6d0444f005099080482dcae5dc22be54cd4724743ec779bba6711c43d5ee7e399d66e679669ef9cf3ebf7d06fc43011fdd4f533a71bd382da4a813a7ae4b32947b41018b5cebb6eeef1d3f72f16fbffaf5793dfce4f1c77ff35cedb9f33a64067ec9818fe5cd32c58e6d1730b53cea60cf26aec77df2108c955cea3c16b93af7b32b0b8f1f4231d7734c2bf3d8962eb0716b4a4d2f3d91d940cf1b9b431fe23a41c75675db0e657030321b38c3fc7933f8d3e650ede8e5b77fc0761fbb367b32b0a53dd77822ccf06150cd7a5ed18df5f51511ea35ecd2b469154dbd57b70b7da4420bb40f413885206cfe8b9232f188d3e7d96e54b7a2a4ec4521441246bc64488452216d286949d2d322510824484784f01a85bd93459af972dab10b5f1421829280a0a09e61aa60bce1bee15d8e569ab1ba6d126621d3b7dcd2d70802ce38c4a0384fcb348fcb1246b0b76865beee15f3e5a89d9aec350b2443c3dd910864063e15063d9f58fcfe538b6766eabf98a92f9c46b0be70ba76f468fde727e7998ed02cd3fdd930b3c0bcca407082f9daba4ee61483c1feebf6857bb0bf146f5c24d3cb4ef8298c2046622b0f9d914ec85c606a00fca51d6c5c7c76ae76f689fa8923dded61b6e7c57670890db1dc45167c7535fed3c472f57c29757d7532ce9b99ac67d8150b4b2d2671c3bab6aed6ef656f566b9d7be89e47827fb2a1207799058f3eb8e56aad16fce8835bf0ddedaf5d0cadec735f58c5d8c406106c6bfb127760f997bba161f8d4c2b91fbfcc7233ed21863bd20e3aea0ba7cdabf33fea6e0f333dff0a82530d6dcfb120b1b6ed68a9b9c4dab6e0ee36ef2fec2dd5fbe977e90f161a9a7d9e05c9b506a9a592926b0dd3ddadde6fa05aaaf233ef3ae20e2d5e571cbcfcfa6b975fb9f674c737d9b67976a3aaf0b2c60bb2f216abc4c7c4cac864ff407f7f7fffae8989bda2b22f33b237313436ee68db54c518da35f1c823c9b896a866fb0b78f71e7938399631ecb381c09b81ff630e7c3340c1ded53827969eb51ddc881df6480697f9168ee1d681ae9ecefafccc95f3734d3fb341e654b0c91c97823b9bc17a58d5bd6482227b64fa40bc42c8cea11d3495a6de5425572d233b3e319e4dc2ddd0328431a3343831902a5813767a746cb0e059bbb769f9dc1033ca4cccfe8efd0e13e07cbef940f38d9f92d73f25fb10f9002172157ce3a7eefb4fdd3e1cde031cfa7cf3dee49bf0efd951e687ccbf39f0f1e5e9dd5219328bdc432b75a137be7162f1a56f470ea158d1a1e9953a11efd7897c8ef239cae7289fa37c14f0eb447e9de883c8513d9f0b75715d37a38eca5c603e023616a671dacee7ed0ab7a1f6ad7357cf3dd3c3869867984078e57e0ebcbae91daef79edc045e0bbec3f5dea9e36fcdfffd8eebbda7830fe07aeff5c09dd8f6c700b81408d59f78f90fdfbd7691e9fee9d9b75fdab4a5b376ec4cedd917aebc78b236772cc20c57c1d45af8cd8c421509584e0b342da78921688a487455e591206b905745cd40446d4d6f0d807a9ee9ded400a8b60b0c00a19e0d20186ae3da96ce4757356b1a25d332b325626aea6d87c4ac56656747b9b82f3b8a73c2a83838bcdd8be71e1e15f68c1cec9fdcee0d0d25154293467c687ce990fc0f757a4b8857e4baa7e8c8b99d0707c64b245920c37b93b9e1ec417597551dd99feedf95b2c5c9b2e51db0b6a752396744db7730571a3c80c50445037a263a98c1e2cd654d7f5fdf3ffbbafc39570baec7e79cff5e78dfeb67f5696c250f71635e14c9b22a88b220485016b01355902c891aaf89bc262009a98aa861238a70298a70b6d92a8bb2a4421949a22c4111eb59238af56c298a1d438fa2f06247e4af3f7965e6b7f0f3ff61cd86882d92c7aea3c78a2537eb52a74c9dc3169df2b05e725cdb89adc31c0e575c1737dc358f9269c496068992a64950e425110afc4d9d8ca50e48819a067959b9d190b7338dc12a442a9214248a2a12244d51c5669f34f5f42c2eb8b15b16706378cb96829bc1de7491c69cc315c77071aeecc696712161db8504b172cbc0b01217a84189d74409a908dfd475cc239e7b7b5f81474841488248f8ca878798ff020000ffff010000ffffe0a2c71089240000}

Popular Tag

Related