Not able to sync with git sha 6837001

General
1

Without the networking and verifier improvements, I wasn’t able to sync to tip.

Here’s an indication of my previous syncing speed:

logs/min2-1748603039.log:I (11:39:42) block 4oiKXnikLoawTuUfvtAhQaqb7Q7pukUs3jwp3WxSNvjqjK16Y29zgZ9 added to validated blocks at 684
logs/min2-1748603039.log:I (11:42:49) block 74z3QBmozseYDrtPc3KJyGDYL51e3psaw8mkcPBvSP4XFYAjnA6ybP6 added to validated blocks at 685
logs/min2-1748603039.log:I (11:45:54) block C5zuxid7kN566hzxaQV5c79BLDVggJ4V56jiFrZ1mQP1A5RJmeY4kEW added to validated blocks at 686
logs/min2-1748603039.log:I (11:49:11) block 3hxcJ13wiKLcSqoJEWsBB5ZhnArpdJ4vZFFTYeP85zWx26ZDvu7CY6C added to validated blocks at 687
logs/min2-1748603039.log:I (11:52:31) block 2J4QzyvknLoDXvCttSjCUWrGF1adSkrRKrBdAUKVFdq7jrcZN9WQPpd added to validated blocks at 688
logs/min2-1748603039.log:I (11:55:39) block Biv7PNftyhV2J26juHb8jCQ5c8NRKjJEA7j3non7thqBYs5v3qEYAxA added to validated blocks at 689
logs/min2-1748603039.log:I (11:59:12) block 98z7XXLi8GW9L9tX3H3oRPa8f5PkyqEczogSKTq6MPQTJ3u8zmGknst added to validated blocks at 690
logs/min2-1748603039.log:I (12:02:33) block 61g4U29Pf9coBpDrq4muoWzqRm4Xm6FwXemjEBMjo5NRcoYWU5qgUr7 added to validated blocks at 691
logs/min2-1748603039.log:I (12:05:39) block 9jnkU4XuTBgiRdvBWLTWpsYc9uF8vd2FVt9dnQ6hpYEj5mAimZzTCJ added to validated blocks at 692
logs/min2-1748603039.log:I (12:08:42) block tKqLFXQv3xDip8U7zUxAvxXCx9oPGUDzuLFGq8v5miiFgJpzAbLeQW added to validated blocks at 693
logs/min2-1748603039.log:I (12:12:46) block fN4zfWrV1c4rSDqKSv6S25UR7FaqTJpdFYcyeWs44T3XE1JV2KN76K added to validated blocks at 694
logs/min2-1748603039.log:I (12:15:44) block CaaT6LQMZb3LnRJiCdyM3V1pm1mDEmcBLw1hz4SevjQ3gwb3H3g7Uo5 added to validated blocks at 695
logs/min2-1748603039.log:I (12:19:01) block 83pSTkyJ74nMsYXqokaKGf4bLugevt4YQceuQ1bdCmS4JzE9bXee48L added to validated blocks at 696
logs/min2-1748603039.log:I (12:26:03) block AwWc1o8myCVR8igUYVZtWBDhFwofPBZW1sG2aw1xZ54fo6fgcgRZswh added to validated blocks at 697
logs/min2-1748603039.log:I (12:30:18) block 8N8usGSeW1T2xYvQq5j4ewEUb2g5S7Xhp9psHSi9x64EqppMqTwYSsz added to validated blocks at 698
logs/min2-1748603039.log:I (12:34:43) block 9uVhEHtkuiMvFApGuF7H4b654ZV4hoJ74zcGwXPLALNyhhoVYVGhVRh added to validated blocks at 699
logs/min2-1748603039.log:I (12:41:58) block 6xW3GLKmmXyAxWVehZsvJPLifzwRg2vTPpproC3VY2fF2QYehH3nt5u added to validated blocks at 700

After building and running the latest code, I lost a bunch of previously verified blocks, and sync is now fast but only for the first 15 or 16 blocks.

This is what the init with the new binary looked like. No indication of state needing a reset, other then perhaps “kernel hash changed”. If I read +check-checkpoints correctly, previous consensus state may be dropped, but it would lose it all and log an info lvl message.

I (12:55:41) [no] kernel::boot: kernel: starting
I (12:55:41) [no] kernel::form: Found existing state - restoring from checkpoint
I (12:55:45) [
I (12:55:45) [no] kernel::form: Kernel hash changed: Hash("26d2e3187cb697644950792844266187258e1543ac848c7d6dbb82f8af54aafb") -> Hash("ff3b6000ab0415a67e55ce7ef4c577e7e4efdf5f1cb78226476d45b31ebdc196")) nockchain: Loaded identity as peer 12D3KooWE2fgsRZ8iK35aSs7bos1MpEXbgM1JAwQziT2aedUyRBg
allowed_peers_path: None
I (12:55:45) driver_init: driver 'bitcoin_watcher' initialized
I (12:55:45) libp2p_swarm: local_peer_id=12D3KooWE2fgsRZ8iK35aSs7bos1MpEXbgM1JAwQziT2aedUyRBg
I (12:55:45) driver_init: driver 'libp2p' initialized
I (12:55:45) nc: SEvent: Listening on /ip4/127.0.0.1/udp/45000/quic-v1
I (12:55:45) nc: SEvent: Listening on /ip4/37.27.8.56/udp/45000/quic-v1
I (12:55:45) nc: Failed to bootstrap: No known peers.
I (12:55:45) nc: SEvent: Listening on /ip4/172.17.0.1/udp/45000/quic-v1
W (12:55:46) behaviour: Failed to trigger bootstrap: No known peers.
I (12:55:46) driver_init: driver 'mining' initialized
I (12:55:46) driver_init: all drivers initialized, born poke sent
I (12:55:47) dumbnet born
I (12:56:06) nc: SEvent: friendship ended with 12D3KooWHywnc9u2EW5AZeoWnB287V8YHKUezUQZMuPNXBtDZ7NU via: Dialer { address: /ip4/34.162.206.28/udp/30000/quic-v1/p2p/12D3KooWHywnc9u2EW5AZeoWnB287V8YHKUezUQZMuPNXBtDZ7NU, role_override: Dialer, port_use: Reuse }. cause: Some(IO(Custom { kind: Other, error: Connection(ConnectionError(ApplicationClosed(ApplicationClose { error_code: 0, reason: b"" }))) }))}. cause: Some(IO(Custom { kind: Other, error: Connection(ConnectionError(ApplicationClosed(ApplicationClose { error_code: 0, reason: b"" }))) }))
I (12:56:35) Ac3MrVRdxR1yyT1j57ByodLCMTf6CMLwP82yUKRLBsdDrrG36cNnEFZ is new heaviest block
I (12:56:35) dumbnet: new heaviest block!
LTragy4tGs7txtNAGbMub1o, role_override: Dialer, port_use: Reuse }. cause: Some(IO(Custom { kind: Other, error: Connection(ConnectionError(ApplicationClosed(ApplicationClose { error_code: 0, reason: b"" }))) }))
I (12:56:55) generating new candidate block with parent: 8eTC6Gsen9qoKisYGxeN8v16A4xiY813Tovd1ocNrPLfGwoTqZvsmy
I (12:57:03) nc: SEvent: friendship ended with 12D3KooWBq7z1Ur4faewRtzaiwMkqLTragy4tGs7txtNAGbMub1o via: Dialer { address: /ip4/51.75.54.23/udp/3006/quic-v1/p2p/12D3KooWBq7z1Ur4faewRtzaiwMkqLTragy4tGs7txtNAGbMub1o, role_override: Dialer, port_use: Reuse }. cause: Some(IO(Custom { kind: Other, error: Connection(ConnectionError(ApplicationClosed(ApplicationClose { error_code: 0, reason: b"" }))) }))
I (12:57:03) nc: SEvent: friendship ended with 12D3KooWHywnc9u2EW5AZeoWnB287V8YHKUezUQZMuPNXBtDZ7NU via: Dialer { address: /ip4/34.162.206.28/udp/30000/quic-v1/p2p/12D3KooWHywnc9u2EW5AZeoWnB287V8YHKUezUQZMuPNXBtDZ7NU, role_override: Dialer, port_use: Reuse }. cause: Some(IO(Custom { kind: Other, error: Connection(ConnectionError(ApplicationClosed(ApplicationClose { error_code: I (12:56:55) generating new candidate block with parent: 8eTC6Gsen9qoKisYGxeN8v16A4xiY813Tovd1ocNrPLfGwoTqZvsmy
I (12:57:03) nc: SEvent: friendship ended with 12D3KooWBq7z1Ur4faewRtzaiwMkqLTragy4tGs7txtNAGbMub1o via: Dialer { address: /ip4/51.75.54.23/udp/3006/quic-v1/p2p/12D3KooWBq7z1Ur4faewRtzaiwMkqLTragy4tGs7txtNAGbMub1o, role_override: Dialer, port_use: Reuse }. cause: Some(IO(Custom { kind: Other, error: Connection(ConnectionError(ApplicationClosed(ApplicationClose { error_code: 0, reason: b"" }))) }))
I (12:57:04) block 72rjh4BC89uyxKUGxGQMWUvupWtvXMrZGvJv5tEoFaEWdmnLQbtDyRg added to validated blocks at 367
I (12:57:04) 72rjh4BC89uyxKUGxGQMWUvupWtvXMrZGvJv5tEoFaEWdmnLQbtDyRg is new heaviest block
I (12:57:04) dumbnet: new heaviest block!

Here’s the new syncing speed, with a couple of restarts (different log files):

logs/min2-1748789127.log:I (14:47:21) block BAhKE5TJ5c8a1TAZXtqjGeq6zUo6kBH6vopV6hZeS5AwZftxALz37Ap added to validated blocks at 455
logs/min2-1748789127.log:I (14:47:31) block Ci5vUHobXWPc9tzmhDWXay1uiD4R2bUT7gsxDyW2zMXbYnTWRnPJFzh added to validated blocks at 456
logs/min2-1748789127.log:I (14:47:44) block 8G2yN9f58nKKkAXgJ4nUgD4DZkChKTUk3czpA1YYbSDmriPtwxRgHh8 added to validated blocks at 457
logs/min2-1748789127.log:I (14:48:06) block 6FQLd6MhUkvAwhWEUJoF772Qm6w5cQCTS1GiJwx2v53tSDzXi8UGpMd added to validated blocks at 458
logs/min2-1748789127.log:I (14:48:41) block 791WNnJ9DTK4xS53B9c2Dzrj4RpkvFyCfnMBmhmoaBqiXbiW6NkxVuG added to validated blocks at 459
logs/min2-1748789127.log:I (14:49:45) block A4WTUgDDRne4jezwv95hJLtwCkWWhzSRhS279N3YEvS67M6w2fwUmSQ added to validated blocks at 460
logs/min2-1748789127.log:I (14:55:50) block iv5LQ6fpUNwdmDtqUEEfv4E7WExkGi5XFjEWMpuCjirc6CGjhjRq3J added to validated blocks at 461
logs/min2-1748791544.log:I (15:25:59) block 9fp1Pm5jYxVH6zs9LZvdzR6Kz85mTBCHM8BT8AB9rtXuS3pUHwSnGGr added to validated blocks at 462
logs/min2-1748791544.log:I (15:26:09) block 6YyRfyGUmr8rs7chBqT2yZuNzT8boG6Jmge6rJ3Q7jQk8pHgy8reAEu added to validated blocks at 463
logs/min2-1748791544.log:I (15:26:19) block AZqVEu338ikWKncX8K8cRJjvra9ZxHP4q6TJAbwnWGnnQD6g1BZZoCe added to validated blocks at 464
logs/min2-1748791544.log:I (15:26:29) block C6DxTbbZR5ToZ5m3zbCMWvMsVch9NjqSHQGV5Abwzro2JgGZHf2FU7b added to validated blocks at 465
logs/min2-1748791544.log:I (15:26:39) block Ad9iBC5y9AghUUwxnighQt5J6QStS9VMtvCHXPo3BAUKb9bH6Cd9P2n added to validated blocks at 466
logs/min2-1748791544.log:I (15:26:50) block 2yRn9Z1tzULaf2zDGYdn51hXtQbr2zScwpcfyBTwppwYHNtbqMuu7WD added to validated blocks at 467
logs/min2-1748791544.log:I (15:27:00) block 7byCQb5WJaN2yWCd4yxcWyjjhKe4N5xXius7dZAsLzQt7RAaVHCiN2U added to validated blocks at 468
logs/min2-1748791544.log:I (15:27:11) block 79LzRSgSE2T8cYaAZQS1bGyfCRUcepS9VwWkp468Y4BfTsKYqyqaS4Z added to validated blocks at 469
logs/min2-1748791544.log:I (15:27:21) block 6rMCCNKuqLGHwncYUAzB4JJwuJoa5A1n8qD6E6xvu4d1yPm3Gv1m7XG added to validated blocks at 470
logs/min2-1748791544.log:I (15:27:37) block 81nRTUSh1DzuVnsZYGRehdbsdheD5kanhQTTWWCQvNVt9R4iQ9CpSot added to validated blocks at 471
logs/min2-1748791544.log:I (15:27:49) block 2S1uyFSQMA9WfodbNKmqfGSM9YU7KiuB3eFPtQcvSng5rQhDPSn9XKt added to validated blocks at 472
logs/min2-1748791544.log:I (15:28:07) block AjQuFZ8K6yeGQ9BwCVi4oepK4mUhQcDkkkXaaguNmnUW8QQQXXNRQed added to validated blocks at 473
logs/min2-1748791544.log:I (15:28:28) block DYZMQ2u9ZUZLWVxgwNZmB5TWG8EccoYazke659qYhR7CpKV119m2KW6 added to validated blocks at 474
logs/min2-1748791544.log:I (15:28:48) block A4PhVeUNKJEiZJ2GRT3CLRWCSHLzm8CFM8j9jKMSZrAqv6UCkzqjzZr added to validated blocks at 475
logs/min2-1748791544.log:I (15:29:08) block WNNoKAuJWLpSeMvRTBb6CVkwvRcy2Fr1eHcJ812oxXtCr48c8cGk53 added to validated blocks at 476
logs/min2-1748791544.log:I (15:29:35) block 7dWpN42eSp2AZ5TkjdgTCf9DkwfSaiX8uV46Tvi2f9aXkfuSWay3rVq added to validated blocks at 477
logs/min2-1748791544.log:I (15:30:04) block 9jBVDsdpXo6yrg8WUnX5dyJsL6dmuhtNMmqTiUtoHUQnJknaLMTcREs added to validated blocks at 478
logs/min2-1748791544.log:I (15:31:14) block DefmdTpEZuKvLMz1P76gNvkRu48Gr6KzLPoJ4r5Tcj7fjv6VhzGzP8J added to validated blocks at 479
logs/min2-1748792201.log:I (15:36:56) block 8UUKQUZHiE8JTpvwFfpNoENV85WrmX66h4mGdn6aB4TXK4EuL8R3rMT added to validated blocks at 480
logs/min2-1748792201.log:I (15:37:10) block 99Cbh6wvaJFoU46gMfVxwLAiYBtCTKvQbp6k8kosN6GAjLRaGo8DYag added to validated blocks at 481
logs/min2-1748792201.log:I (15:37:20) block 9QTpVz8BWiHyBMGkhzH5EavvjFkNT9NAFQvjCxBSjwLeob7Dz6ZTSNG added to validated blocks at 482
logs/min2-1748792201.log:I (15:37:30) block 9buJ5ycxbyhiNd1o5xh7SeZyryUwGcSV8hsQkrTJXf6Z5QLTnehaAkq added to validated blocks at 483
logs/min2-1748792201.log:I (15:37:40) block 275xGLbZbaSjR2HNhZYyJDckVF8YPbNBm4sCxnmRfNAp29NKG15fYV8 added to validated blocks at 484
1 Like
2

Workaround for now, pick your poison:

#!/bin/bash

COMMAND="./your-command"   # replace with your actual command
DURATION=5m                # or use 300s

while true; do
    echo "Starting process at $(date)..."
    timeout --preserve-status "$DURATION" $COMMAND
    echo "Process ended at $(date), restarting after short delay..."
    sleep 1
done

import subprocess
import time
import signal

# Configuration
COMMAND = ["./your-command"]  # Replace with your command and args
DURATION = 5 * 60             # Run time in seconds

def run_once():
    print(f"Starting process at {time.strftime('%Y-%m-%d %H:%M:%S')}")
    proc = subprocess.Popen(COMMAND)

    try:
        proc.wait(timeout=DURATION)
    except subprocess.TimeoutExpired:
        print(f"Time limit reached. Sending SIGTERM to process (PID {proc.pid})...")
        proc.terminate()

        try:
            proc.wait(timeout=10)  # Give it 10 seconds to shut down cleanly
        except subprocess.TimeoutExpired:
            print("Process didn't exit in time. Killing it with SIGKILL...")
            proc.kill()
            proc.wait()

    print(f"Process exited with code {proc.returncode}\n")

if __name__ == "__main__":
    try:
        while True:
            run_once()
            time.sleep(1)  # Short pause between restarts
    except KeyboardInterrupt:
        print("Loop interrupted by user.")