Migrating stateful apps to K8s 1.29 — persistent volume issues

Hey everyone, we're in the middle of moving a legacy PostgreSQL + Redis stack to Kubernetes 1.29 and running into some rough edges with PV provisioning.

Setup: Hetzner Cloud, 3 control plane nodes, 6 workers. We're using Longhorn for storage backend, but PVCs are getting stuck in "Pending" after node reboots. The events log just says "waiting for first consumer" but nothing is actually consuming them.

I've checked:

• StorageClass exists and has allowVolumeExpansion: true
• CSI driver pods are running
• No quota restrictions
• Enough free space

Anyone dealt with this? Is there a networking or CNI thing I'm missing? Our Cilium setup is fairly vanilla. Trying to avoid a full cluster nuke-and-rebuild if possible lol

Thanks in advance!

Check if your Longhorn replicas are actually getting scheduled post-reboot. We hit this exact issue — PVCs stuck in Pending because Longhorn couldn't place replicas across the cluster. Try kubectl get longhorn-replicas -A and see if they're in pending state.

Also verify your StorageClass has numberOfReplicas set appropriately for your node count. With 6 workers, 3 replicas is safe, but if you're using anti-affinity rules without enough nodes, they'll never schedule. Run kubectl describe pvc <your-pvc> and look at the actual error events — sometimes K8s is vague but Longhorn logs tell the real story.

Thanks for the pointer! Just checked and yeah, looks like the Longhorn replicas aren't spreading across nodes after reboot. Running kubectl get nodes shows some workers still in NotReady state. Going to investigate the kubelet logs and see if there's a CSI sync issue. Cheers!