Secondary storage migration error (Stats Collector)

[bordeldecul]

Hello,

I'm trying to move the secondary storage to a new server and get this strange error (CS version 4.19.1.1):

Stats Collector hasn't yet collected metrics from the Image store, kindly try again later

(localcloud) 🐱 > migrate secondarystoragedata srcpool=b326cef5-cd07-4a02-83fc-e93ead32e7af destpools=7d79d220-b0d6-4812-bca5-32be68be6e3d migrationtype=complete { "account": "admin", "accountid": "358e091e-cae2-11e3-b855-000c2991ca67", "cmd": "org.apache.cloudstack.api.command.admin.storage.MigrateSecondaryStorageDataCmd", "completed": "2024-09-06T17:34:29+0200", "created": "2024-09-06T17:34:28+0200", "domainid": "248fcada-cae2-11e3-b855-000c2991ca67", "domainpath": "ROOT", "jobid": "d88608bd-0004-4854-a5cb-cfec819f0df8", "jobinstancetype": "ImageStore", "jobprocstatus": 0, "jobresult": { "errorcode": 530, "errortext": "Stats Collector hasn't yet collected metrics from the Image store, kindly try again later" }, "jobresultcode": 530, "jobresulttype": "object", "jobstatus": 2, "userid": "358e1dbe-cae2-11e3-b855-000c2991ca67" } 🙈 Error: async API failed for job d88608bd-0004-4854-a5cb-cfec819f0df8

Old Secondary storage is now read only and the new one is read/write.
Any suggestion on how to debug this ?

[Pearl1594]

You will need to wait for some time until the stats collector runs to get details of the capacity of the destination store (newly added one). As per the code, the Storage stats collector runs every minute.

[bordeldecul]

I've been waiting quite some time now and still get the error.
Is there a way to now when the stats collector has collected the details ? (if it runs at all)
Is there a command to force it to run ?

[Pearl1594]

In the Management server logs, do you see any of the following logs:

1. StorageCollector is running...
or
2. There is no secondary storage VM for secondary storage host

If the 2nd log is present, it's most likely skipping the store. Could you provide the following information to help further:

1. Does the zone in question have an SSVM?
2. Do both the image stores have the URL field populated as it could be another reason why the stats collector has skipped the store (https://github.com/apache/cloudstack/blob/main/server/src/main/java/com/cloud/server/StatsCollector.java#L1673-L1674)

[bordeldecul]

Thank you for the help.
The error message was misleading.
I was able to narrow it down to a network issue.
The SSVM is in a new IP range and cannot mount the old secondary storage.