> For the complete documentation index, see [llms.txt](https://docs.bitplanet.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bitplanet.ai/node-operation/troubleshooting.md).

# Troubleshooting

## Common Issues

### Node Not Syncing

**Symptoms:**

* `catching_up: true` persists
* Block height not increasing
* No peers connected

**Diagnosis:**

```bash
# Check sync status
curl -s http://localhost:26657/status | jq '.result.sync_info'

# Check peers
curl -s http://localhost:26657/net_info | jq '.result.peers | length'

# View logs
sudo journalctl -u evmd -f --output cat | grep -i error
```

**Solutions:**

```bash
# 1. Check persistent peers configuration
grep persistent_peers $HOME/.evmd/config/config.toml

# 2. Add more peers
PEERS="<node_id>@<ip>:26656,<node_id2>@<ip2>:26656"
sed -i.bak "s/^persistent_peers *=.*/persistent_peers = \"$PEERS\"/" $HOME/.evmd/config/config.toml

# 3. Restart node
sudo systemctl restart evmd

# 4. If still not syncing, use state sync (faster)
# Get trust height and hash from a synced node
TRUST_HEIGHT=$(curl -s http://rpc-endpoint:26657/block | jq -r '.result.block.header.height')
TRUST_HEIGHT=$((TRUST_HEIGHT - 1000))  # Use height ~1000 blocks ago for safety
TRUST_HASH=$(curl -s http://rpc-endpoint:26657/block?height=$TRUST_HEIGHT | jq -r '.result.block_id.hash')

echo "Trust Height: $TRUST_HEIGHT"
echo "Trust Hash: $TRUST_HASH"

# Edit config.toml with these values
[statesync]
enable = true
rpc_servers = "http://rpc1.example.com:26657,http://rpc2.example.com:26657"
trust_height = $TRUST_HEIGHT
trust_hash = "$TRUST_HASH"
trust_period = "168h0m0s"  # 7 days (must be less than unbonding period of 21 days)

# Then restart the node
sudo systemctl restart evmd
```

### Validator Jailed

**Symptoms:**

* Validator status shows `jailed: true`
* Not signing blocks
* Alerts triggered

**Diagnosis:**

```bash
# Note: Set VALCONS if not already set (see "Checking Validator Status" section)
# Check validator status
evmd query staking validator $(evmd keys show validator --bech val -a --keyring-backend file --home $HOME/.evmd) \
  --node http://localhost:26657 | jq '{jailed, status, tokens}'

# Check signing info
evmd query slashing signing-info $VALCONS \
  --node http://localhost:26657
```

**Solutions:**

```bash
# 1. Ensure node is fully synced
curl -s http://localhost:26657/status | jq '.result.sync_info.catching_up'
# Should return: false

# 2. Unjail validator
evmd tx slashing unjail \
  --from validator \
  --keyring-backend file \
  --home $HOME/.evmd \
  --node http://localhost:26657 \
  --chain-id 9001 \
  --gas auto \
  --gas-adjustment 1.5 \
  --gas-prices 10000000bplcoin \
  --yes

# 3. Verify unjailing
evmd query staking validator $(evmd keys show validator --bech val -a --keyring-backend file --home $HOME/.evmd) \
  --node http://localhost:26657 | jq '.jailed'
```

### Genesis Compatibility Issues

**Symptoms:**

* Node fails to start with genesis validation errors
* Errors like `invalid CreatorSplit: invalid decimal string`
* Genesis parameter format incompatibilities

**Diagnosis:**

```bash
# Validate genesis file
evmd genesis validate-genesis --home $HOME/.evmd

# Check genesis hash
sha256sum $HOME/.evmd/config/genesis.json

# Compare with running nodes
curl -s http://rpc-endpoint:26657/genesis | jq -r '.result.genesis' | sha256sum
```

**Solutions:**

```bash
# 1. Verify you're using the correct evmd binary version
evmd version

# 2. Re-download genesis from a trusted source
curl -s http://rpc-endpoint:26657/genesis | jq -r '.result.genesis' > $HOME/.evmd/config/genesis.json

# 3. If genesis sync fails, use state sync instead
# See "Node Not Syncing" section above for state sync configuration

# 4. Ensure genesis file matches the network
GENESIS_HASH=$(sha256sum $HOME/.evmd/config/genesis.json | awk '{print $1}')
echo "Your genesis hash: $GENESIS_HASH"
echo "Compare with network's genesis hash"
```

## Consensus Failure Recovery

### Double Sign Prevention

**If you suspect double signing:**

```bash
# IMMEDIATELY stop the validator
sudo systemctl stop evmd

# Check if validator key is compromised
# Compare priv_validator_key.json across all your nodes
# THERE SHOULD ONLY BE ONE ACTIVE NODE WITH THIS KEY

# If running multiple nodes by mistake:
# 1. Stop ALL nodes
# 2. Identify the correct, synced node
# 3. Start ONLY that one node
# 4. Delete priv_validator_key.json from all other nodes

# Verify no double signing occurred
# Note: Set VALCONS if not already set (see "Checking Validator Status" section)
evmd query slashing signing-info $VALCONS \
  --node http://localhost:26657
```

### State Corruption

**Symptoms:**

* Node crashes on startup
* Database errors in logs
* Inconsistent state errors

**Recovery:**

```bash
# 1. Stop node
sudo systemctl stop evmd

# 2. Reset tendermint state (keeps application state)
evmd tendermint unsafe-reset-all --home $HOME/.evmd

# 3. If corruption persists, restore from snapshot
rm -rf $HOME/.evmd/data
tar -xzf evmd-state-backup.tar.gz -C $HOME/.evmd/data

# 4. Restart node
sudo systemctl start evmd

# 5. Monitor sync
sudo journalctl -u evmd -f
```

## Log Analysis

### Useful Log Patterns

```bash
# View real-time logs
sudo journalctl -u evmd -f --output cat

# Filter for errors only
sudo journalctl -u evmd -f | grep -i error

# Check for specific issues
sudo journalctl -u evmd --since "1 hour ago" | grep -i "connection refused"

# View consensus logs
sudo journalctl -u evmd -f | grep "consensus"

# Check block production
sudo journalctl -u evmd -f | grep "committed state"
```

## Next Steps

* Review setup procedures in [Setup & Configuration](/node-operation/setup-configuration.md)
* Learn about daily operations in [Operations](/node-operation/operations.md)
* Follow step-by-step procedures in [Runbooks](/node-operation/runbooks.md)
* Access quick reference in [Additional Resources](/node-operation/additional-resources.md)