...
Scale down the Runtime and Listener deployments in the PROD environment by running the two commands in the following formats.
Code Block language css theme Midnight kubectl scale --replicas=0 deployment <Release Name>-ac-runtime -n <namespace>
...
Code Block language css theme Midnight kubectl scale --replicas=0 deployment <Release Name>-ac-listener -n <namespace>
...
Pause Scheduler in the PROD environment
https://<webapp_Gateway_URL>/event/schedulerservice?_dc=1670999141899&query=Pause&page=1&start=0&limit=25
...
Scale down Rabbit MQ statefulset in the PROD environment
kubectl
...
scale
...
--replicas=0
...
statefulset
...
<Release
...
Name>-ac-rabbitmq
...
-n
...
<namespace>
...
- Scale down Rabbit MQ statefulset in the DR environment
kubectl
...
scale
...
--replicas=0
...
statefulset
...
<Release
...
Name>-ac-rabbitmq
...
-n
...
<namespace>
...
Initiate Failover for the storage account using below steps:
...
Info >> These failover steps are not mandatory. These steps are mandatory only when the Azure File Share is not accessible due to any failure.
>> As per Microsoft guideline, the time it takes to failover after initiation can vary though typically less than one hour.
Login into your Azure Portal.
...
Go to the Storage Account being used in the PVC of Prod environment.
...
Within that storage account go to Data Management > Redundancy section.
...
Click Prepare for failover and follow the instruction given on the screen. The failover process starts.
...
Once the Account Failover process completes the data from secondary region will be accessible from the existing end point of the storage account
...
Note:
...
.
...
...
As per Microsoft guideline, the time it takes to failover after initiation can vary though typically less than one hour.
- Copy web/repository folder from PROD to DR environment
./azcopy
...
sync
...
"https://[account].file.core.windows.net/[PVC]/web/repository/?[SAS]"
...
"https://[account].file.core.windows.net/[PVC]/web/repository/?[SAS]"
...
--recursive=true
...
...
- Example:
...
./azcopy
...
sync
...
...
...
--recursive=true
...
...
Copy recovery folder from PROD to DR environment
./azcopy
...
sync
...
"https://[account].file.core.windows.net/[PVC]/recovery/?[SAS]"
...
"https://[account].file.core.windows.net/[PVC]/recovery/?[SAS]"
...
--recursive=true
...
...
Example:
...
./azcopy
...
sync
...
...
...
--recursive=true
...
- Copy reprocessing folder from
...
- PROD to DR environment
./azcopy
...
- sync
...
- "https://[account].file.core.windows.net/[PVC]/reprocessing/?[SAS]"
...
- "https://[account].file.core.windows.net/[PVC]/reprocessing/?[SAS]"
...
- --recursive=true
...
For
...
- Example:
...
./azcopy
...
- sync
...
- "https://fxxxxxxxxxxxxxxxxxxxxx1b6a.file.core.windows.net/pvc-e38eca50-xxxx-xxxx-xxxx-828efd6d49df/web/reprocessing/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2024-03-06T01:18:24Z&st=2023-03-05T17:18:24Z&spr=https,http&sig=VxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxaXBmSAKm4%3D"
...
- "https://fxxxxxxxxxxxxxxxxxxx3aeb.file.core.windows.net/pvc-1a6f1c1b-xxxx-xxxx-xxxx-44ff485d707f/reprocessing/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2024-03-06T01:25:19Z&st=2023-03-05T17:25:19Z&spr=https,http&sig=vxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxGazWA%3D"
...
- --recursive=true
...
- Copy Rabbitmq PVC content from PROD to DR environment
./azcopy
...
- sync
...
- "https://[account].file.core.windows.net/[PVC]/?[SAS]"
...
- "https://[account].file.core.windows.net/[PVC]/?[SAS]"
...
- --recursive=true
...
- --mirror-mode=true
...
For
...
- Example:
...
./azcopy
...
- sync
...
- "https://fxxxxxxxxxxxxxxxxxxxxx1b6a.file.core.windows.net/pvc-ef345e9d-xxxx-xxxx-xxxx-54e38de9cbd1/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2024-03-06T01:18:24Z&st=2023-03-05T17:18:24Z&spr=https,http&sig=VxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxaXBmSAKm4%3D"
...
- "https://fxxxxxxxxxxxxxxxxxxx3aeb.file.core.windows.net/pvc-4a47fa22-xxxx-xxxx-xxxx-f6538dfff1a7/?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2024-03-06T01:25:19Z&st=2023-03-05T17:25:19Z&spr=https,http&sig=vxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxGazWA%3D"
...
- --recursive=true
...
- --mirror-mode=true
...
- Scale up RabbitMQ statefulset in the DR environment
kubectl
...
- scale
...
- --replicas=1
...
- statefulset
...
- <Release
...
- Name>-ac-rabbitmq
...
- -n
...
- <namespace>
...
- Scale up Listener in the DR environment
kubectl
...
- scale
...
- --replicas=1
...
- deployment
...
- <Release
...
- Name>-ac-runtime
...
- -n
...
- <namespace>
...
- Resume Scheduler in the DR environment
https://<webapp_Gateway_URL>/event/schedulerservice?_dc=1670999141899&query=Resume&page=1&start=0&limit=25
...
- Update DNS record to route all request to DR environment
Switching back from DR to PROD Environment
Scale down Runtime and Listener deployments in the DR environment.
...
Pause Scheduler in the DR environment
...
Scale down Rabbit MQ statefulset in the DR environment
...
Scale down Rabbit MQ statefulset in the PROD environment
...
Copy web/repository folder from DR to PROD environment
...
Copy recovery folder from DR to PROD environment
...
Copy reprocessing folder from DR to PROD environment
...
Copy Rabbitmq PVC content from DR to PROD environment
...
Scale up RabbitMQ statefulset in the PROD environment.
...
Scale up Listener in the PROD environment
...
Resume Scheduler in the PROD environment
...
Update DNS record to route all request to PROD environment
Using DR to recover from a failure
...