Question
Given your experience of observing bitcoin p2p and stale rates, what tooling or extra data in p2p messages you wish bitcoin had that will give you better insights into latency, stale rates, compact block miss rates, etc?
At p2poolv2 we want to make it easier to observe the network by providing the tooling or change data included in p2p messages to make observability better without leaking information about nodes.
Background
We are working on compact block relays for P2Poolv2 to send shares for our first version. In contrast, the original p2pool used a custom protocol that eliminated the round trips required with compact blocks to hydrate blocks with missing txs.
The original p2pool protocol maintained state on each peer connection. The state tracked which transactions the peer on the other end has definitely seen. When sending a share to a peer, the sending node checks which txs the peer has not seen, and precede the share message with remember_tx messages. Thus each node sends missing txs before the share.
The protocol has the downside of maintaining state, but the plus side of eliminating round trip times to fetch missing compact block transactions.
We are not sure we want to build a stateful protocol first. So we are building compact blocks for our mvp and later depending on miss rates and other observed data, debate changing the protocol.
Rough Estimates
We are aiming for 10s share interval. Largely cause bandwidths have improved since original p2pool, we are using uncles to reward stales, plus hopefully our Rust implementation is faster than the original p2pool in python.
uncle_rate_increase = miss_rate × RTT / share_interval
Assuming median RTT of 300ms, and say 3% of transactions missing from compact blocks, we get to uncle_rate_increase = 0.03 * 300ms / 10000ms = 0.09%
This is a small increase, but every increase adds up. If we see miss rate more than 3%, we might have to think of something else.
So what can we add to p2p messages or headers to improve observability? ![]()
Also, is there a compact block miss rate observation tool somewhere? I see this: Calculate and expose the stale block rate per network · Issue #94 · 0xB10C/fork-observer · GitHub