Two block reorg at height 941880

We just had a rare-ish two block reorg where Foundry mined a couple of blocks in a row winning a two block long race against AntPool and ViaBTC.

Foundry ended up mining seven blocks in a row:

The stale blocks are:

00000000000000000001b3ff8b13e57c3ec1eca3ba7d2937edbd9f219eb2d9f3 (AntPool) at height 941881
00000000000000000000c81cbf94a12ca498e72eb8530f7061c8746cf9687b2e (ViaBTC) at height 941882

While Foundry mined the following blocks to win the race:

00000000000000000000bd4930a5982911e7749eb491886206e71abdc1ec0cc6 (Foundry) at height 941881
00000000000000000000724eac69a18c6699c9f7aaab24bcf18beb2723ccadd2 (Foundry) at height 941882
000000000000000000009c9acd0bc3207fa181f79f8573bf27d8a81d1ef3aa8e (Foundry) at height 941883
4 Likes

Can you share the timestamps on those blocks or even the headers if possible? Am wonder if this was a result of network latency?

Welcome!

I just added the headers and full blocks to the stale-block dataset: add: stale blocks for 941881 & 941882 by 0xB10C · Pull Request #90 · bitcoin-data/stale-blocks · GitHub

Will check the arrival times on those four too.

1 Like

You can see them on fork.observer, just click the blocks for details:

Stale chaintip:
941881 — 1774280975
941882 — 1774281079

Best chain:
941881 — 1774280987
941882 — 1774281085

Note that the timestamps only need to be accurate within ±2h, so more interesting would be the arrival times instead of the timestamps.

3 Likes

I found it interesting that boerst’s stratum-work did not see any stratum jobs from Foundry mining on their winning blocks. See e.g. 941882 on stratum.work.

Maybe they were only mining on them in one region, and stratum-work is connected to a different one, or they kept quite about them (i.e. selfish mining), or it’s a bug on stratum-work :slight_smile:

1 Like

Yup, I am just hoping these pool operators are running sensible clocks - if not using ntp.

I looked into this cause I am interested in the uncles probability in p2poolv2. If we look at the fork monitor, it looks like if blocks are within 15-30 seconds of each other it results in a fork. This makes me think if we can get the mean/stddev on the fork blocks timestamps deltas Again, assuming clock drift is not too high.

It’ll be fascinating to see how this delta changes between different pairs of pools. Detect friends and enemies :slight_smile:

There is data on block arrival times and block header timestamps in GitHub - bitcoin-data/block-arrival-times: A public Bitcoin Block Arrival Time Dataset licensed under CC0. · GitHub if you want to compare it over time.

@willcl-ark recently shared this plot with me showing 12s-21s deltas on average:

(which makes sense since most pools send out new jobs to miners in 30s intervals)

1 Like

Yeah, but they should also send a new job immediately on seeing a new block. Unless of course there are some strategies they use to detect how to react to blocks from different pools. Might be interesting to uncover :slight_smile:

Thanks for that link to the data. We don’t see the pool name there, right? Do you think we can include that for the data from now on? Or it might be in another data set somewhere that I can join. We want to get the pool names for the orphan blocks too.

Sorry, am new to all your work, so probably asking naive questions.

EDIT: Just this: Offer: Stale block dataset

Per the stale-block repo, previous known two block stale chains:

  • 788687 / 788686 - 2023-05-08 ; 153k blocks ago
  • 656478 / 656477 - 2020-11-12 ; 132k blocks earlier
  • 358999 / 358998 - 2015-06-02 ; 300k blocks earlier
  • 261199 / 261198 - 2013-10-02 ; 98k blocks earlier

So perhaps every ~3 years or 150k blocks?

I’m not sure what the maths would be to figure out what to expect there – naively I would have thought you’d square the frequency of one-block reorgs (since you need two blocks in a row to be found almost simultaneously to avoid everyone agreeing on a single tip) which would lead to expecting about a million blocks between 2-block reorgs if we’re seeing 1-block reorgs every thousand blocks, but perhaps you get different results if you model internal delays getting jobs/work distributed between the pool and ASICs separately from network block propagation.

My timings for the headers in question were:

2026-03-23T15:49:53.982265Z Saw new cmpctblock header hash=00000000000000000001b3ff8b13e57c3ec1eca3ba7d2937edbd9f219eb2d9f3 height=941881 peer=14899 peeraddr=45.32.83.173:54302
2026-03-23T15:51:47.008245Z Saw new header hash=00000000000000000000c81cbf94a12ca498e72eb8530f7061c8746cf9687b2e height=941882 peer=38718 peeraddr=124.156.199.113:36184
2026-03-23T15:55:03.500660Z Saw new header hash=000000000000000000009c9acd0bc3207fa181f79f8573bf27d8a81d1ef3aa8e height=941883 peer=36252 peeraddr=65.109.99.249:35498

I didn’t see Foundry’s racing blocks until the reorg. (Which might suggest that 1-block stale blocks are much more common than 1-in-a-thousand, but they just aren’t visible)

FWIW, I’m not collecting Foundry’s templates on https://stratum.work/ - I could never get a working account from them.

From the pools that I do collect, I can see that none of the other pools were sending work building on top of Foundry’s 941881 and 941882 blocks. Everyone else was seemingly following the AntPool/ViaBTC chain.

2 Likes

Thanks for clarifying. My Foundry stratum endpoint doesn’t seem to work either. I and I don’t see Foundry listed on mempool.space/stratum either.

1 Like

I looked a bit at the header and block arrival times in the debug.logs of my monitoring nodes. Here, “header” means we either got an INV for this block, saw a cmpctblock or a header. Full block means we received the block or successfully reconstructed the block from a compact block.

                               node           block       event
timestamp                                                      
2026-03-23 15:49:53.864840      ian  941881 AntPool      header
2026-03-23 15:49:53.903500     dave  941881 AntPool      header
2026-03-23 15:49:53.965877     jade  941881 AntPool      header
2026-03-23 15:49:53.972246      ian  941881 AntPool  full_block
2026-03-23 15:49:53.976826     mike  941881 AntPool      header
2026-03-23 15:49:54.029989     owen  941881 AntPool      header
2026-03-23 15:49:54.057771     jade  941881 AntPool  full_block
2026-03-23 15:49:54.133473     owen  941881 AntPool  full_block
2026-03-23 15:49:54.239330  charlie  941881 AntPool      header
2026-03-23 15:49:54.241502      bob  941881 AntPool      header
2026-03-23 15:49:54.256592     nico  941881 AntPool      header
2026-03-23 15:49:54.350779    frank  941881 AntPool      header
2026-03-23 15:49:54.356334    alice  941881 AntPool      header
2026-03-23 15:49:54.400475     greg  941881 AntPool      header
2026-03-23 15:49:54.446445     erin  941881 AntPool      header
2026-03-23 15:49:54.457890      bob  941881 AntPool  full_block
2026-03-23 15:49:54.582114    hazel  941881 AntPool      header
2026-03-23 15:49:54.600324  charlie  941881 AntPool  full_block
2026-03-23 15:49:54.604217     mike  941881 AntPool  full_block
2026-03-23 15:49:54.612627     nico  941881 AntPool  full_block
2026-03-23 15:49:54.715069     dave  941881 AntPool  full_block
2026-03-23 15:49:54.764564     greg  941881 AntPool  full_block
2026-03-23 15:49:54.812221     kane  941881 AntPool      header
2026-03-23 15:49:54.824742     luke  941881 AntPool      header
2026-03-23 15:49:54.893326     erin  941881 AntPool  full_block
2026-03-23 15:49:55.037321    hazel  941881 AntPool  full_block
2026-03-23 15:49:55.238127    frank  941881 AntPool  full_block
2026-03-23 15:49:55.440619     luke  941881 AntPool  full_block
2026-03-23 15:49:56.627557    alice  941881 AntPool  full_block
2026-03-23 15:49:57.215005     kane  941881 AntPool  full_block
2026-03-23 15:51:47.052497     nico   941882 ViaBTC      header
2026-03-23 15:51:47.058206    alice   941882 ViaBTC      header
2026-03-23 15:51:47.061556  charlie  941881 Foundry      header
2026-03-23 15:51:47.069931     luke   941882 ViaBTC      header
2026-03-23 15:51:47.078155     greg   941882 ViaBTC      header
2026-03-23 15:51:47.079686     jade   941882 ViaBTC      header
2026-03-23 15:51:47.082866      bob  941881 Foundry      header
2026-03-23 15:51:47.083817    alice   941882 ViaBTC  full_block
2026-03-23 15:51:47.085359      ian   941882 ViaBTC      header
2026-03-23 15:51:47.087298     mike   941882 ViaBTC      header
2026-03-23 15:51:47.092817  charlie   941882 ViaBTC      header
2026-03-23 15:51:47.098123     erin   941882 ViaBTC      header
2026-03-23 15:51:47.101483    frank   941882 ViaBTC      header
2026-03-23 15:51:47.102551     jade   941882 ViaBTC  full_block
2026-03-23 15:51:47.103210      ian   941882 ViaBTC  full_block
2026-03-23 15:51:47.110371      bob   941882 ViaBTC      header
2026-03-23 15:51:47.118183     owen   941882 ViaBTC      header
2026-03-23 15:51:47.128487     nico   941882 ViaBTC  full_block
2026-03-23 15:51:47.130324     kane   941882 ViaBTC      header
2026-03-23 15:51:47.132378     greg   941882 ViaBTC  full_block
2026-03-23 15:51:47.133369     mike   941882 ViaBTC  full_block
2026-03-23 15:51:47.152286     owen   941882 ViaBTC  full_block
2026-03-23 15:51:47.160858     erin   941882 ViaBTC  full_block
2026-03-23 15:51:47.163834  charlie   941882 ViaBTC  full_block
2026-03-23 15:51:47.175300     kane   941882 ViaBTC  full_block
2026-03-23 15:51:47.195791    hazel   941882 ViaBTC      header
2026-03-23 15:51:47.238768     dave   941882 ViaBTC      header
2026-03-23 15:51:47.244357      bob   941882 ViaBTC  full_block
2026-03-23 15:51:47.247765    frank   941882 ViaBTC  full_block
2026-03-23 15:51:47.263158    hazel   941882 ViaBTC  full_block
2026-03-23 15:51:47.536036     dave  941881 Foundry      header
2026-03-23 15:51:47.767039     greg  941881 Foundry      header
2026-03-23 15:51:48.526856     dave   941882 ViaBTC  full_block
2026-03-23 15:51:50.377779     luke   941882 ViaBTC  full_block
2026-03-23 15:55:00.641682      bob  941883 Foundry      header
2026-03-23 15:55:01.176160  charlie  941882 Foundry      header
2026-03-23 15:55:01.191329     dave  941882 Foundry      header
2026-03-23 15:55:01.370924      bob  941882 Foundry  full_block
2026-03-23 15:55:01.370934     greg  941882 Foundry      header
2026-03-23 15:55:01.429284     owen  941882 Foundry      header
2026-03-23 15:55:01.711011     greg  941883 Foundry      header
2026-03-23 15:55:01.714383      ian  941883 Foundry      header
2026-03-23 15:55:01.778498    alice  941883 Foundry      header
2026-03-23 15:55:01.780852     owen  941883 Foundry      header
2026-03-23 15:55:01.816236      bob  941883 Foundry  full_block
2026-03-23 15:55:02.773480     mike  941883 Foundry      header
2026-03-23 15:55:03.360435  charlie  941883 Foundry      header
2026-03-23 15:55:03.491469     owen  941883 Foundry  full_block
2026-03-23 15:55:03.542388      ian  941881 Foundry  full_block
2026-03-23 15:55:03.698902     luke  941883 Foundry      header
2026-03-23 15:55:03.716347     nico  941881 Foundry      header
2026-03-23 15:55:03.716464     nico  941882 Foundry      header
2026-03-23 15:55:03.716509     nico  941883 Foundry      header
2026-03-23 15:55:04.374432  charlie  941883 Foundry  full_block
2026-03-23 15:55:04.456908     dave  941883 Foundry      header
2026-03-23 15:55:04.973663     jade  941883 Foundry      header
2026-03-23 15:55:05.439823    frank  941883 Foundry      header
2026-03-23 15:55:05.886777     erin  941881 Foundry      header
2026-03-23 15:55:05.886891     erin  941882 Foundry      header
2026-03-23 15:55:05.886936     erin  941883 Foundry      header
2026-03-23 15:55:06.085654     dave  941883 Foundry  full_block
2026-03-23 15:55:06.374803     erin  941881 Foundry  full_block
2026-03-23 15:55:06.747076     erin  941882 Foundry  full_block
2026-03-23 15:55:06.801136     mike  941881 Foundry  full_block
2026-03-23 15:55:07.071938     erin  941883 Foundry  full_block
2026-03-23 15:55:07.172630    hazel  941883 Foundry      header
2026-03-23 15:55:07.231769     nico  941881 Foundry  full_block
2026-03-23 15:55:07.351077     mike  941882 Foundry  full_block
2026-03-23 15:55:07.901623     mike  941883 Foundry  full_block
2026-03-23 15:55:08.888249     nico  941882 Foundry  full_block
2026-03-23 15:55:09.346723    hazel  941881 Foundry  full_block
2026-03-23 15:55:09.965627     jade  941881 Foundry  full_block
2026-03-23 15:55:10.133469    hazel  941882 Foundry  full_block
2026-03-23 15:55:10.326101     nico  941883 Foundry  full_block
2026-03-23 15:55:10.670198    hazel  941883 Foundry  full_block
2026-03-23 15:55:10.681589     kane  941883 Foundry      header
2026-03-23 15:55:11.418276      ian  941882 Foundry  full_block
2026-03-23 15:55:13.304834     greg  941883 Foundry  full_block
2026-03-23 15:55:14.318127     jade  941882 Foundry  full_block
2026-03-23 15:55:16.414858      ian  941883 Foundry  full_block
2026-03-23 15:55:17.683597     jade  941883 Foundry  full_block
2026-03-23 15:55:17.998145    alice  941881 Foundry  full_block
2026-03-23 15:55:19.254253     luke  941881 Foundry  full_block
2026-03-23 15:55:20.573447     kane  941881 Foundry  full_block
2026-03-23 15:55:22.239878    alice  941882 Foundry  full_block
2026-03-23 15:55:24.322924    alice  941883 Foundry  full_block
2026-03-23 15:55:25.850183    frank  941881 Foundry  full_block
2026-03-23 15:55:27.524423     kane  941882 Foundry  full_block
2026-03-23 15:55:28.616653     luke  941882 Foundry  full_block
2026-03-23 15:55:29.847000    frank  941882 Foundry  full_block
2026-03-23 15:55:31.190630    frank  941883 Foundry  full_block
2026-03-23 15:55:32.942391     kane  941883 Foundry  full_block
2026-03-23 15:55:34.202208     luke  941883 Foundry  full_block
2026-03-23 16:01:48.751482  charlie  941881 Foundry  full_block
2026-03-23 16:01:49.014432     dave  941881 Foundry  full_block
2026-03-23 16:01:49.573302  charlie  941882 Foundry  full_block
2026-03-23 16:01:50.206303     dave  941882 Foundry  full_block
2026-03-23 16:01:54.667334      bob  941881 Foundry  full_block
2026-03-23 16:05:03.395825     owen  941881 Foundry  full_block
2026-03-23 16:05:04.331536     greg  941881 Foundry  full_block
2026-03-23 16:05:04.384588     owen  941882 Foundry  full_block
2026-03-23 16:05:05.058914     greg  941882 Foundry  full_block

I created the following plot from this.

My interpretation of this is:

  • I received the headers and blocks for 941881 AntPool (red) on all nodes at roughly the same time.
  • I did not hear about a header or block of 941881 Foundry (blue).
  • Then, the ViaBTC block and Foundry block 941882 (green) were mined.
  • I learned about ViaBTCs header and block (orange) on all nodes at roughly the same time. I did not learn about Foundry block 941882 (green).
  • At the same time, I learned about 941881 Foundry (blue) headers on node greg, bob, charlie, dave. I did not learn about the full block.
  • At a later time, Foundry 941883 (purple, the winning block) was mined and I learned about he Foundry headers and blocks
  • On some nodes, I only learned about the full Foundry blocks >5min later.
1 Like

Foundry released their 1st block (header) immediately after the others’ 2nd block was found. According to their own timestamp on their 1st and 2nd blocks, they found them a few seconds after the others’ 1st and 2nd blocks. They released their 2nd and 3rd at the same time, exactly when the 3rd block was found, winning the race. This is effectively a selfish-mining attack by definition even if it was on “accident” or network hiccup (to the extent “attack” doesn’t imply intentional). But it all seems to point to intentional.

Intentional or not, other miners should ignore 2 of Foundry’s blocks. Without enforcing accurate timestamps, this how selfish mining can be made a “nothing burger”. >50% needs to collude to disincentivize future attacks. If Foundry stops displaying it’s name to prevent this, the only solution is to enforce more accurate timestamps. For example, the rule would be to ignore a header for 600 seconds if its timestamp is more than 15 seconds different from local time (rounded to the nearest 10 s interval) when it arrives. This works because the attacker has to assign a timestamp before he knows when he will need to release it. People complain that this can cause a real problem in real network partitions and it’s true a chain split would last longer if that’s the case, but it will eventually resolve itself based on PoW and which partition has the most hashrate. Rounding to the nearest 10 second interval prevents an attack on the interval. Clocks shouldn’t be off more than 2 seconds which is the other big complaint. Also, miners would have to update the timestamp every 2 seconds.

Concerning not being able to get Foundry’s templates, Grok says:

Yes, restricting public or easy access to block templates can serve Foundry (or any large pool) if they were engaging in illicit or strategically deviant behavior like selfish mining or block withholding—primarily by reducing detectability and transparency.

(post deleted by author)