Sadeeq raised a potential issue with time-based expiry of mempool transactions whereby a transaction that is likely to be mined in the next block could be evicted. David responded with a great history of time-based mempool expiry. After reading the thread, I’m not particularly convinced we need to index by time, but I would like to know how many transactions are expired due to the default timeout of 2 weeks.
I came across this issue when researching what it would take to remove the boost multi-index from Bitcoin Core dependencies. No matter the approach, removing time as a key would greatly simplify the patch-set.
This is probably derivable from the other data offers, but I thought I would post this for discussion anyway.
I noticed that we log two lines for each expiring transaction, so just grepping for a reason=expiry will yield two log entries for each transaction that expires.
While analyzing the data, it took me a while to find something that conveys what I’m seeing: The upper part shows the average number of expiries per day on the nodes that saw at least one expiry on that day. This is usually somewhere between 10 and 10k. There are also days without any expiries. This is shown in the bottom part of the chart: when it reaches 100%, all nodes didn’t see an expiry.
I was surprised to see only very few days between 2025-02-01 and 2025-09-15 with expires. I see the other obvious mempool removal reasons on these dates (block, conflict, …) but no reason=expiry.
For both batches, all transactions I looked at were eventually mined. So either someone rebroadcast them, or pools kept them around. The first batch transactions all seem to reveal an inscription and have a runestone OP_RETURN. The second batch all seem to have a OP_RETURN runestone, but don’t reveal an inscription.
I was wondering if these transactions were rebroadcast or if the miners still had them. We don’t know for sure what the miners did, but grepping for the txids show that they all were re-added to my mempool on 2025-10-03.
Thanks for the comprehensive research, certainly an interesting result. Given the example of the mass expiry with a big wave of rebroadcast, it seems the expiry mechanism is just adding marginal bandwidth use for no gain, as they were all eventually mined. I wonder if we can find examples of 1. a transaction is “expired,” was not seen in a later rebroadcast announcement, but was later mined, indicating miners are running a longer expiry. 2. a transaction was “expired” and was in fact replaced by adding/changing inputs. (1) is not possible to prove with certainty of course, but if a new tx announcement does not come up in a well-connected node’s logs and yet the (expired) tx was mined, I think we can have high confidence the default expiry was not used by miners. In investigating (1) if it was found that all transactions end up being re-broadcast, that would indicate to me that time isn’t doing much at all in practice.
