In our latest deployment we're going from Migration v72 to v81. We failed on v73 with

error: canceling statement due to statement timeout

Looking closer, we see that specifically the creation of an index on

ExplanationOfBenefit_Token

failed. This is the largest table by row count in our database. It was somewhat counter-intuitive to us that the timeout applies to

CONCURRENTLY

created indices, but it appears to be the case. We see that there is a default connection setting timeout here: https://github.com/medplum/medplum/blob/b31ff6cf06ccd060b325c1a22d1ad73db15cd62e/packages/server/src/database.ts#L66C15-L75 Seeing this config, is the intended deployment pattern for migrations on large databases to disable the client configuration during the migrations and then re-apply it later?

Hey @joshua_kelly - thanks for sending, apologies for the inconvenience. Can you share what server version you're using? Timeouts should be disabled on migrations (see https://github.com/medplum/medplum/blob/main/packages/server/src/database.ts#L97). So clearly something didn't work as expected there. > Seeing this config, is the intended deployment pattern for migrations on large databases to disable the client configuration during the migrations and then re-apply it later? The migrations should be re-entrant, so you should be able to try again, however we may want to take a more hands on approach. Feel free to DM if you'd like to chat directly. I'd be happy to help navigate.

@cody I'm trying this again now - one thing I can say is that this exact query took 10 minutes last time:

CREATE INDEX CONCURRENTLY IF NOT EXISTS 
ExplanationOfBenefit_Token_code_value_idx
 ON ExplanationOfBenefit_Token ( code, value ) INCLUDE ( resourceId )

Ok, interesting. Is it still running? If you hit a timeout again, let us know, and we can try to dig in and reproduce the error.

A little more investigation revealed the issue - we have circuit breaker deploys set up ECS to fail deploys at 15 minutes After 15 minutes ECS killed the deployment This is probably not what you have setup in the CDK - we are managing the ECS config directly with the rest of our stack