You can use a CI/CD pipeline for importing the Adeptia Connect objects to a new environment automatically. This CI/CD pipeline uses the Exported ZIP file that you must have created as a prerequisite for import operation, and imports the objects to the target environment.
...
The diagram below represents how the import pipeline works, and automates the import process.
When triggered, the pipeline performs the following sequence of actions.
...
To start with importing the objects, you first need to create an import pipeline having all the required parameters for exporting the objects.
Info |
---|
Adeptia provides you with |
...
an import pipeline file that you can |
...
use to create the pipeline. |
After you create the pipeline, you need to trigger the pipeline to perform the import operation.
...
To create the pipeline in Jenkins using the import pipeline provided by Adeptia, follow the steps given below.
Log in to the Jenkins with admin privileges.
Select New Item.
Enter a name for the Import pipeline, and then select Pipeline.
Click OK.
Copy the content the content from the provided import provided import pipeline file.
Expand title Import pipeline Code Block language css theme Midnight import jenkins.model.Jenkins import java.nio.file.Path; import java.nio.file.Paths; import java.io.File; /* This pipeline used to deploys migration promotion through operations like import, rollback, retain from one build environment to another. Pipeline is made up of following steps 1. Init Perameters 2. Pull import/retain XML and source solution zip file from GitHub 3. Upload import/retain XML and source solution zip to k8 shared PVC 4. Download Helm chart & deploy migration solution (import/retain/rollback) 5. Download rollback zip from k8 shared PVC 6. Push rollback zip to GitHub 7. Clean up workspace Pre-requisite a) Tools/Plugins needs to install: 1. Helm 2. kubectl client 3. Jenkins 4. Java 1.8+ b) OS Linux c) Jenkins plugins 1. Kubernetes 2. Nexus 3. Git (git plugin 4.8.3, Git client plugin 3.9.0) 43. Mask Password 54. Credentials Binding Plugin (1.27) 65. Parameter Separator 76. BlueOcean (Optional) Usage: Steps to create pipeline using jenkinsfile. 1. Login into the Jenkins GUI with admin privileges. 2. create a pipeline by choosing New Item > Pipeline. 3. Copy/past containt of jenkinsfile to Pipeline Definition area. 4. Uncheck checkbox "Use Groovy Sandbox". 5. Save the pipeline. 6. Trigger the pipeline once to initilaize parameters. */ /* Upload file to Kubernetes PVC */ def uploadToSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, SRC_FILE_PATH, TRG_FILE_PATH) { echo "Upload file("+SRC_FILE_PATH+")files to K8 shared PVC" withKubeConfig([credentialsId: K8_CREDENTIALS_ID, serverUrl: SERVER_URL]) { try { wrap([$class: 'MaskPasswordsBuildWrapper', varPasswordPairs: [[NEXUS_PASSWORD:'NEXUS_PASSWORD']]]) { sh ''' #!/bin/shbash kubectl config use-context '''+CLUSTER_CONTEXT+''' TRG_FILE_PATH='''+TRG_FILE_PATH+''' if [[ ${TRG_FILE_PATH::1} == "/" ]] then TRG_FILE_PATH=${TRG_FILE_PATH:1}; else echo "Forward shash(/) already removed "; fi podname=$(kubectl -n '''+NAMESPACE+''' get pods | grep -m 1 autoscaler | awk '{print $1}') kubectl -n '''+NAMESPACE+''' cp '''+SRC_FILE_PATH+''' ${podname}:${TRG_FILE_PATH} jobname=$(kubectl -n '''+NAMESPACE+''' get jobs | grep -m 1 migration | awk '{print $1}') if [[ -n "$jobname" ]]; then kubectl -n '''+NAMESPACE+''' delete job ${jobname} else echo "Migration resource does not exist" fi ''' } } catch (err) { echo "Caught: ${err}. Error in uploading file." error("Caught: ${err}") currentBuild.result = 'FAILURE' } } } /* Download file from Kubernetes PVC */ def downloadFromSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, SRC_FILE_PATH, TRG_FILE_PATH) { echo "Download file("+SRC_FILE_PATH+")files from K8 shared PVC" withKubeConfig([credentialsId: K8_CREDENTIALS_ID, serverUrl: SERVER_URL]) { try { sh ''' #!/bin/shbash SRC_FILE_PATH='''+SRC_FILE_PATH+''' if [[ ${SRC_FILE_PATH::1} == "/" ]] then SRC_FILE_PATH=${SRC_FILE_PATH:1}; else echo "Forward shash(/) already removed "; fi kubectl config use-context '''+CLUSTER_CONTEXT+''' podname=$(kubectl -n '''+NAMESPACE+''' get pods | grep -m 1 autoscaler | awk '{print $1}') kubectl -n '''+NAMESPACE+''' cp ${podname}:${SRC_FILE_PATH} '''+TRG_FILE_PATH+''' ''' } catch (err) { echo "Caught: ${err}. Error in downloading file from K8 PVC." error("Caught: ${err}") currentBuild.result = 'FAILURE' } } } /* Pull Helm Chart */ def pullHelmChart (NEXUS_CREDENTIALS_ID, NEXUS_HELM_HELM_REPO_URL, CHART_NAME) { echo "Pull Helm Chart ("+CHART_NAME+") from NexusArtifact repositoryHub" withCredentials([usernamePassword(credentialsId: NEXUS_CREDENTIALS_ID, passwordVariable: 'NEXUS_PASSWORD', usernameVariable: 'NEXUS_USERNAME')]) { try { sh '''#!/bin/hide password field bash wrap([$class: 'MaskPasswordsBuildWrapper', varPasswordPairs: [[NEXUS_PASSWORD:'NEXUS_PASSWORD']]]) { sh 'helm repo add adeptia-connect-migration '''+HELM_REPO_URL+''' #!/bin/sh helm repo add nexushelmrepo '''+NEXUS_HELM_REPO_URL+''' --username '''+NEXUS_USERNAME+''' --password '''+NEXUS_PASSWORD+''' helm pull nexushelmrepopull adeptia-connect-migration/'''+CHART_NAME+''' --untar ''' } } catch (errcatch (err) { echo "Caught: ${err}. Error in pulling Helm chart from Nexus repo." error("Caught: ${err}") currentBuild.result = 'FAILURE' } } } /* Deploy Helm to Kubernetes cluster */ def deployToCluster (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, DATABASE_CREDENTIALS_ID, SERVER_URL) { echo "Deploy Helm chart to Kubernetes cluster" try { def BACKEND_DB_USERNAME = getUserName(DATABASE_CREDENTIALS_ID); def BACKEND_DB_PASSWORD = getPassword(DATABASE_CREDENTIALS_ID); withKubeConfig([credentialsId: K8_CREDENTIALS_ID, serverUrl: SERVER_URL]) { //hide password field wrap([$class: 'MaskPasswordsBuildWrapper', varPasswordPairs: [[password:BACKEND_DB_PASSWORD], [password:BACKEND_DB_USERNAME]]]) { sh ''' #!/bin/shbash kubectl config use-context '''+CLUSTER_CONTEXT+''' helm upgrade -i migration migration -f migration/config/values-qa.yaml --set environmentVariables.BACKEND_DB_URL=${BACKEND_DB_URL} --set environmentVariables.BACKEND_DB_USERNAME='''+BACKEND_DB_USERNAME+''' --set environmentVariables.BACKEND_DB_PASSWORD='''+BACKEND_DB_PASSWORD+''' --set environmentVariables.BACKEND_DB_DRIVER_CLASS=${BACKEND_DB_DRIVER_CLASS} --set environmentVariables.BACKEND_DB_TYPE=${BACKEND_DB_TYPE} --set environmentVariables.SOURCE_ZIP_PATH=${SOURCE_ZIP_PATH} --set environmentVariables.MIGRATION_XML_FILE_PATH=${MIGRATION_XML_FILE_PATH} --set environmentVariables.OVERRIDE_USER=${OVERRIDE_USER} --set environmentVariables.OVERRIDE_MODIFIEDBY_USER=${OVERRIDE_MODIFIEDBY_USER} --set environmentVariables.RETAIN_XML_PATH=${RETAIN_XML_PATH} --set environmentVariables.LOG_IDENTIFIER=${LOG_IDENTIFIER} --set environmentVariables.OPERATION=${OPERATION} --set image.tagenvironmentVariables.ROLLBACK_ZIP_PATH=${LATESTROLLBACK_ZIP_TAGPATH} -n '''+NAMESPACE+''' ''' } } } catch (err) { echo "Caught: ${err}. Error in deploying Helm chart." error("Caught: ${err}") currentBuild.result = 'FAILURE' } } /* Wait until deployment finish on Kubernetes cluster */ def waitUntilDepoymentComplete(NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, POD, time_out) { echo "Try to fetch the Fetching pod status" try { int inter = 5, count = 1; withKubeConfig([credentialsId: K8_CREDENTIALS_ID, serverUrl: SERVER_URL]) { sh('kubectl config use-context ${CLUSTER_CONTEXT};') while (true) { def status = sh script: "kubectl -n ${NAMESPACE} get pods | grep -m 1 ${POD} | awk '{print \$3}' ", returnStdout: true if (status.toString().trim().contains("Completed")) { break; } else if (status.toString().trim().contains("Error")) { error("Caught: Migration deployment failed due to error. Please check migration logs.") currentBuild.result = 'FAILURE' sleep(inter) echo count+" retry in "+inter*count+" seconds." count++ if ((count)>=((time_out-5)/inter)) { error("Caught: Migration deployment is taking more then ideal time. Please check migration logs.") currentBuild.result = 'FAILURE' break; break; } } } } catch sleep(errinter) { echo count+"Caught: ${err}. Error in fetching pod status." retry in "+inter*count+" seconds." count++ if ((count)>=((time_out-10)/inter)) { error("Caught: ${err} Migration deployment taking more then ideal time. Please check migration logs.") currentBuild.result = 'FAILURE' } }break; /* } Push soution Zip to GitHub reposirory */ def pushToGitHub (GIT_BRANCH, GIT_CREDENTIALS_ID, GIT_REPO_URL, FILE_PATH) { echo "Push file ("+FILE_PATH+") to GitHub repo"} withCredentials([gitUsernamePassword(credentialsId: GIT_CREDENTIALS_ID, gitToolName: 'git-tool')]) { } try { sh('sleep 10') } catch (err) { sh('git add '+FILE_PATH) sh('git commit -m "auto commit message" ') sh('git push ${GIT_REPO_URL} HEAD:'+GIT_BRANCH) } catch (err) {echo "Caught: ${err}. Error in fetching pod status. Migration deployment taking more then ideal time. Please check migration logs." echo error("Caught: ${err}. Error in pushing file to Github." error("Caught: ${err}"") currentBuild.result = 'FAILURE' } } } /* GeneratePush rollback soution Zip fileto GitHub pathrepository */ def convertRollbackZipPath(pushToGitHub (GIT_BRANCH, GIT_CREDENTIALS_ID, GIT_REPO_URL, FILE_PATH) { def rollbackZipPath = null def Append = "Rollback_" echo "Pushing file ("+FILE_PATH+") to GitHub repo" withCredentials([gitUsernamePassword(credentialsId: GIT_CREDENTIALS_ID, gitToolName: 'git-tool')]) { try { Pathdef pathgitUser = Paths.get(FILE_PATHgetUserName(GIT_CREDENTIALS_ID); def fileName=path.getFileName().toString() sh('sleep 10') def parentDir=path.getParent().toString() sh('git config --global user.name "'+gitUser+'"') rollbackZipPath=parentDir + File.separator + Append + fileName sh('git config --global user.email "you@example.com"') if(isUnix()){sh('git add '+FILE_PATH) rollbackZipPath=rollbackZipPath.replace("\\", "/" sh('git commit -m "auto commit message" ') sh('git push ${GIT_REPO_URL} HEAD:'+GIT_BRANCH) } catch (err) { echo "Caught: ${err}. Error in generatingpushing rollbackfile soution Zip file pathto Github." error("Caught: ${err}") currentBuild.result = 'FAILURE' } return rollbackZipPath} } /* Generate Getrollback usernamesoution fromZip credentialsfile idpath */ def getUserNameconvertRollbackZipPath(idFILE_PATH) { def userNamerollbackZipPath = null withCredentials([usernamePassword(credentialsId: id, passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) { def Append = "Rollback_" try { userNamePath path = USERNAMEPaths.get(FILE_PATH); } catch (err) { def fileName=path.getFileName().toString() echo "Caught: ${err}. Error in extracting username from "+id+" ."def parentDir=path.getParent().toString() rollbackZipPath=parentDir + File.separator + Append + fileName error("Caught: ${err}") if(isUnix()){ rollbackZipPath=rollbackZipPath.replace("\\", "/") } currentBuild.result = 'FAILURE'} catch (err) { } } return userNameecho "Caught: ${err}. Error in generating rollback soution Zip file path." error("Caught: ${err}") currentBuild.result = 'FAILURE' } return rollbackZipPath } /* Get passwordusername from credentials id */ def getPasswordgetUserName(id) { def passworduserName = null withCredentials([usernamePassword(credentialsId: id, passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) { try { passworduserName = PASSWORD;USERNAME } catch (err) { echo "Caught: ${err}. Error in extracting passwordusername from "+id+" ." error("Caught: ${err}") currentBuild.result = 'FAILURE' } } return passworduserName } /* pipeline {Get //password Globalfrom defaultcredentials variablesid */ def getPassword(id) environment { time_outdef password = 100 } parameters{ //separator(name: 'separator-ce1a9ef5-cd10-4002-a43f-8ae24d9d0bb3', sectionHeader: '''Global Parameters''', sectionHeaderStyle: 'background-color:#eeeee4;font-size:15px;font-weight:normal;text-transform:uppercase;border-color:gray;', separatorStyle: '''font-weight:bold;line-height:1.5em;font-size:1.5em;''') string(defaultValue: '', description: 'Nexus credentials ID configured in Jenkins e.g. nexus_credentialsId', name: 'NEXUS_CREDENTIALS_ID', trim: true) string(defaultValue: '', description: 'Nexus Server URL e.g. https://nexus.adeptia.com:8443/repository/adeptia-internal-helm', name: 'NEXUS_HELM_REPO_URL', trim: true) string(defaultValue: '', description: 'Name of Helm chart to be downloaded from Nexus repository e.g migration', name: 'CHART_NAME', trim: true) null withCredentials([usernamePassword(credentialsId: id, passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) { try { password = PASSWORD; } catch (err) { echo "Caught: ${err}. Error in extracting password from "+id+" ." error("Caught: ${err}") currentBuild.result = 'FAILURE' } } return password } pipeline { // Global default variables environment { //manage deployment status timeout time_out = 300 } parameters{ //separator(name: 'separator-ce1a9ef5-cd10-4002-a43f-8ae24d9d0bb3', sectionHeader: '''GitHubHelm Chart Parameters''', sectionHeaderStyle: 'background-color:#eeeee4;font-size:15px;font-weight:normal;text-transform:uppercase;border-color:gray;', separatorStyle: '''font-weight:bold;line-height:1.5em;font-size:1.5em;''') string(defaultValue: '', description: 'GitHubArtifactHub credentialsHelm IDchart configuredURL in Jenkins e.g. gitCredential_idhttps://adeptia.github.io/adeptia-connect-migration/charts', name: 'GITHELM_CREDENTIALSREPO_IDURL', trim: true) string(defaultValue: '', description: 'GitHub server URLName of Helm chart to be downloaded from ArtifactHub repository e.g. https://github.com/adeptia/migration-defination.git', name: 'GITCHART_REPO_URLNAME', trim: true) string//separator(defaultValuename: '', descriptionseparator-ce1a9ef5-cd10-4002-a43f-8ae24d9d0bb3', sectionHeader: '''GitHub Branch name e.g. mainParameters''', namesectionHeaderStyle: 'GIT_BRANCHbackground-color:#eeeee4;font-size:15px;font-weight:normal;text-transform:uppercase;border-color:gray;', trim: trueseparatorStyle: '''font-weight:bold;line-height:1.5em;font-size:1.5em;''') string(defaultValue: '', description: 'SourceGitHub zipcredentials pathID toconfigured bein downloadedJenkins from GitHub. e.e.g. test/SA_PF.zipgitCredential_id', name: 'GIT_SOURCECREDENTIALS_ZIP_PATHID', trim: true) string(defaultValue: '', description: 'ImportGitHub xmlserver file path to be downloaded from GitHub. URL e.g. test/import.xmlhttps://github.com/adeptia/migration-defination.git', name: 'GIT_MIGRATION_XML_FILE_PATHREPO_URL', trim: true) string(defaultValue: '', description: 'RetainGitHub xml file path to be downloaded from GitHub.Branch name e.g. test/retain.xmlmain', name: 'GIT_RETAIN_XML_PATHBRANCH', trim: true) //separatorstring(namedefaultValue: 'separator-ce1a9ef5-cd10-4002-a43f-8ae24d9d0bb3', sectionHeader: '''Migration Parameters''', sectionHeaderStyle: 'background-color:#eeeee4;font-size:15px;font-weight:normal;text-transform:uppercase;border-color:gray;', separatorStyle: '''font-weight:bold;line-height:1.5em;font-size:1.5em;''') string(defaultValue: '', description: 'Location of retain xml file. e.g. export, import or rollback'', description: 'Source zip path to be downloaded from GitHub. e.g. test/SA_PF.zip', name: 'GIT_SOURCE_ZIP_PATH', trim: true) string(defaultValue: '', description: 'Rollback zip path to upload file to GitHub. e.g. test/Rollback_SA_PF.zip. No need to fill this field in case of performing "rollback" operation', name: 'OPERATIONGIT_ROLLBACK_ZIP_PATH', trim: true) string(defaultValue: '', description: 'MigrationImport sourcexml zipfile path. to be downloaded from GitHub. e.g. /shared/test/SA_PFimport.zipxml', name: 'SOURCE_ZIPGIT_MIGRATION_XML_FILE_PATH', trim: true) string(defaultValue: '', description: 'MigrationRetain import/retain xml file path to be downloaded from GitHub. e.g. /shared/test/importretain.xml', name: 'MIGRATIONGIT_RETAIN_XML_FILE_PATH', trim: true) string//separator(defaultValuename: 'separator-ce1a9ef5-cd10-4002-a43f-8ae24d9d0bb3', descriptionsectionHeader: '''User Id or User name with which all objects will be deployed e.g IndigoUser:127000000001107055536473900001Migration Parameters''', sectionHeaderStyle: 'background-color:#eeeee4;font-size:15px;font-weight:normal;text-transform:uppercase;border-color:gray;', separatorStyle: '''font-weight:bold;line-height:1.5em;font-size:1.5em;''') string(defaultValue: '', description: 'Location of retain xml file. e.g. import or rollback', name: 'OVERRIDE_USEROPERATION', trim: true) string(defaultValue: '', description: 'UserMigration Idsource orzip User name which will be reflected in the modified by field of every activity after deployment path. e.g. IndigoUser:127000000001107055536473900001/shared/SA_PF.zip', name: 'OVERRIDESOURCE_MODIFIEDBYZIP_USERPATH', trim: true) string(defaultValue: '', description: 'LocationMigration of import/retain xml file path. e.g. /shared/migration/import/RETAIN_FILENAME.xml', name: 'RETAINMIGRATION_XML_FILE_PATH', trim: true) string(defaultValue: '', description: 'Migration log identifier to capture logs in ms environment.User Id or User name with which all objects will be deployed e.g IndigoUser:127000000001107055536473900001', name: 'LOGOVERRIDE_IDENTIFIERUSER', trim: true) //separator(name: 'separator-ce1a9ef5-cd10-4002-a43f-8ae24d9d0bb3', sectionHeader: '''K8 Cluster Parameters''', sectionHeaderStyle: 'background-color:#eeeee4;font-size:15px;font-weight:normal;text-transform:uppercase;border-color:gray;', separatorStyle: '''font-weight:bold;line-height:1.5em;font-size:1.5em;''') string(defaultValue: '', description: 'Credentials ID configured in Jenkins to access K8 cluster.string(defaultValue: '', description: 'User Id or User name which will be reflected in the modified by field of every activity after deployment e.g. IndigoUser:127000000001107055536473900001', name: 'OVERRIDE_MODIFIEDBY_USER', trim: true) string(defaultValue: '', description: 'Location of retain xml file. e.g. /shared/retain.xml', name: 'K8RETAIN_CREDENTIALSXML_IDPATH', trim: true) string(defaultValue: '', description: 'URLLocation toof accessrollback K8zip clusterfile.', name: 'SERVER_URL', trim: true) string(defaultValue: '', description: 'Cluster context to access K8 cluster. e.g. /shared/Rollback_SA_PF.zip. No need to fill this field in case of performing "rollback" operation', name: 'CLUSTERROLLBACK_ZIP_CONTEXTPATH', trim: true) string(defaultValue: '', description: 'K8Migration clusterlog nameidentifier spaceto deploymentcapture wherelogs Connectin microservicesms deployedenvironment.', name: 'NAMESPACELOG_IDENTIFIER', trim: true) string(defaultValue: '', description: 'URL of database backend bind with application.', name: 'BACKEND_DB_URL', trim: true) string(defaultValue: '', description: 'Credentials ID configured in Jenkins to access database.', name: 'DATABASE_CREDENTIALS_ID', trim: true) //separator(name: 'separator-ce1a9ef5-cd10-4002-a43f-8ae24d9d0bb3', sectionHeader: '''K8 Cluster Parameters''', sectionHeaderStyle: 'background-color:#eeeee4;font-size:15px;font-weight:normal;text-transform:uppercase;border-color:gray;', separatorStyle: '''font-weight:bold;line-height:1.5em;font-size:1.5em;''') string(defaultValue: '', description: 'Driver class of database e.g com.microsoft.sqlserver.jdbc.SQLServerDriver.Credentials ID configured in Jenkins to access K8 cluster.', name: 'BACKENDK8_DBCREDENTIALS_DRIVER_CLASSID', trim: true) string(defaultValue: '', description: 'Database typeURL to access K8 cluster e.g SQL-Server.', name: 'BACKEND_DB_TYPE', trim: true) } agent { label 'LinuxAgent' } stages { stage('Init parameters') { steps { script{ //Global parameters env.LATEST_TAG='80686a8' } } } stage('Checkout files from GitHub') { steps { script { echo 'Checkout xml and zip from github' checkout([$class: 'GitSCM', branches: [[name: '*/'+GIT_BRANCH]], extensions: [], userRemoteConfigs: [[credentialsId: GIT_CREDENTIALS_ID, url: GIT_REPO_URL]]]) https://*******-dns-2ce021bb.hcp.eastus.azmk8s.io:443. You can get the server Url from K8 config file.', name: 'SERVER_URL', trim: true) string(defaultValue: '', description: 'Cluster context to access K8 cluster e.g. adeptia-context', name: 'CLUSTER_CONTEXT', trim: true) string(defaultValue: '', description: 'K8 cluster name space deployment where Connect microservices deployed e.g. adeptia', name: 'NAMESPACE', trim: true) string(defaultValue: '', description: 'URL of database backend bind with application.', name: 'BACKEND_DB_URL', trim: true) string(defaultValue: '', description: 'Credentials ID configured in Jenkins to access database.', name: 'DATABASE_CREDENTIALS_ID', trim: true) string(defaultValue: '', description: 'Driver class of database e.g com.microsoft.sqlserver.jdbc.SQLServerDriver.', name: 'BACKEND_DB_DRIVER_CLASS', trim: true) string(defaultValue: '', description: 'Database type e.g SQL-Server.', name: 'BACKEND_DB_TYPE', trim: true) } /* agent { label 'LinuxAgent' } */ agent any stages { } } stage('UploadCheckout files tofrom PVCGitHub') { steps { script { echo 'UploadCheckout import zipxml and xmlzip file to shared PVCfrom github' uploadToSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, GIT_SOURCE_ZIP_PATH, SOURCE_ZIP_PATH) uploadToSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8checkout([$class: 'GitSCM', branches: [[name: '*/'+GIT_BRANCH]], extensions: [], userRemoteConfigs: [[credentialsId: GIT_CREDENTIALS_ID, SERVER_URL,url: GIT_MIGRATION_XML_FILE_PATH, MIGRATION_XML_FILE_PATH) //Condition to handle retain feature within import if((GIT_RETAIN_XML_PATH.contains(".xml")) && (RETAIN_XML_PATH.contains(".xml"))){ uploadToSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, GIT_RETAIN_XML_PATH, RETAIN_XML_PATH) }REPO_URL]]]) } } } stage('PullUpload Helmfiles Chart & Deploy Migrationto PVC') { steps { script { script { echo 'Pull Migration Helm ChartUploading import zip and xml file' pullHelmChartuploadToSharedPVC (NEXUSNAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, NEXUS_HELM_REPO_URL, CHART_NAME) SERVER_URL, GIT_SOURCE_ZIP_PATH, SOURCE_ZIP_PATH) echo 'Deploy Import' deployToCluster uploadToSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, DATABASE_CREDENTIALS_IDGIT_MIGRATION_XML_FILE_PATH, SERVER_URLMIGRATION_XML_FILE_PATH) timeout(time: env.time_out, unit: "SECONDS"//Condition to handle retain feature within import if((GIT_RETAIN_XML_PATH.contains(".xml")) && (RETAIN_XML_PATH.contains(".xml"))){ waitUntilDepoymentCompleteuploadToSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, 'migration-', env.time_out.toInteger())GIT_RETAIN_XML_PATH, RETAIN_XML_PATH) } } } } stage('Download Rollback zip') {} } stage('Pull Helm Chart when& {Deploy Migration') { steps expression{ script { params.OPERATION == 'import' } } steps { echo 'Pulling Helm Chart' pullHelmChart (HELM_REPO_URL, CHART_NAME) echo 'Deploying Helm Chart' deployToCluster (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, DATABASE_CREDENTIALS_ID, SERVER_URL) timeout(time: env.time_out, unit: "SECONDS"){ waitUntilDepoymentComplete(NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, 'migration-', env.time_out.toInteger()) } } } script { } stage('Download Rollback zip') { echo 'Downloadwhen Rollback{ zip file to shared PVC' downloadFromSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, convertRollbackZipPath(SOURCE_ZIP_PATH), convertRollbackZipPath(GIT_SOURCE_ZIP_PATH)) expression { params.OPERATION == 'import' } } } steps { } } stage('Push Rollback Zip') script { when { echo 'Download Rollback zip file to shared expression { params.OPERATION == 'import' }PVC' downloadFromSharedPVC (NAMESPACE, CLUSTER_CONTEXT, K8_CREDENTIALS_ID, SERVER_URL, ROLLBACK_ZIP_PATH, GIT_ROLLBACK_ZIP_PATH) } } steps { } script {} echo stage('Push Rollback Zip to GitHub' pushToGitHub (GIT_BRANCH, GIT_CREDENTIALS_ID, GIT_REPO_URL, convertRollbackZipPath(GIT_SOURCE_ZIP_PATH))') { when { expression { params.OPERATION == 'import' } } } } steps { } } post('Clean-up') { always { echo 'Cleanup workspace' cleanWs() } success script { echo 'PipelinePush succeeded!' } unstable { Rollback Zip to GitHub' echo 'Pipeline unstable :/' } failurepushToGitHub (GIT_BRANCH, GIT_CREDENTIALS_ID, GIT_REPO_URL, GIT_ROLLBACK_ZIP_PATH) } } } } post('Clean-up') { always { echo 'Pipeline Cleanup workspace' cleanWs() } success { echo 'Pipeline succeeded!' } unstable { echo 'Pipeline unstable :/' } failure { echo 'Pipeline failed :(' } } }
- Paste
In the Pipeline Definition section, paste the copied content
in the Pipeline Definition section in Jenkins.Uncheckand uncheck the Use Groovy Sandbox checkbox.
- Click Save.
On the screen that follows, click Build Now.
As you build the pipeline for the very first time, all the parameters get initialized. The Build Now option now changes to Build with Parameters.Click Build with Parameters.
You will see all the parameters and their values inherited from the import pipeline file.
Change the parameter values as per your requirement.
Expand title Click here to expand the list of Import parameters Parameters Value Description Parameters used in Import pipeline Nexus
NEXUS_CREDENTIALS_ID nexus_credentialsId Store Github access credentials in Jenkins. Select credential type "Username and password" and create global credentials in Jenkins. Example:
user: valuelabs
Pass: password
Note: Use credentials binding plugin to store credentials in Jenkins. Select credential type "Username and password" and save it. A Credentials ID can be added or configures in Jenkins. Using credential ID, the credentials can be accessible globally in Jenkins.
NEXUS_HELM_REPO_URL https://nexus.adeptia.com:8443/repository/adeptia-internal-helm Nexus Helm chart repository URL CHART_NAME migration Helm chart name. GitHub GIT_CREDENTIALS_ID gitCredential_id Store GitHub access credentials in Jenkins. Select credential type "Username and password" and create global credentials in Jenkins.
GIT_REPO_URL https://github.com/prathi-adeptia/migration-defination.git GitHub repository URL. GIT_BRANCH main GitHub branch name. GIT_SOURCE_ZIP_PATH test/import.xml Location to pull the import.xml file from GitHub repository. GIT_MIGRATION_XML_FILE_PATH test/SA_PF.zip Location to pull import zip to GitHub repository. GIT_RETAIN_XML_PATH xml/retain.xml Retain xml file path to be downloaded from GitHub. Migration OPERATION import Specify the type of operation for which deployment will be performed. eg. export, import, rollback SOURCE_ZIP_PATH /shared/migrationtest1/SA_PF.zip Import zip path to upload file on shared PVC (K8 cluster) MIGRATION_XML_FILE_PATH /shared/migrationtest1/import.xml Import.xml path to upload file on shared PVC (K8 cluster) OVERRIDE_USER IndigoUser:127000000001107055536473900001 User Id or User name with which all objects will be deployed OVERRIDE_MODIFIEDBY_USER IndigoUser:127000000001107055536473900001 User Id or User name which will be reflected in the modified by field of every activity after deployment RETAIN_XML_PATH /shared/migrationtest1/retain.xml Location of retain xml file. eg. $SHARED_PATH$/migration/import/RETAIN_FILENAME.xml' LOG_IDENTIFIER Test_Identifier_Tag Log identifier to capture logs from MS environment. K8 Cluster K8_CREDENTIALS_ID k8credentials Store K8 cluster access credentials in Jenkins. Select credential type "Secret file" and create global credentials in Jenkins. Example:
user: valuelabs
Pass: Secret file (Here secret file is k8 .config file)
SERVER_URL https://valuelabs-dev-dns-2ce021bb.hcp.eastus.azmk8s.io:443 Kubernetes cluster URL CLUSTER_CONTEXT test-dev Kubernetes cluster context NAMESPACE namespace Kubernetes cluster namespace for deployment. Database BACKEND_DB_URL jdbc:sqlserver://adeptia-adeptiaconnectdb.database.windows.net:1433;database=valuelabs-backend-8 Backend database URL used in the deployment of migration Helm chart. Database info configured in value.yaml file. DATABASE_CREDENTIALS_ID databasecredentialid Store database access credentials in Jenkins. Select credential type "Username and password" and create global credentials in Jenkins. BACKEND_DB_DRIVER_CLASS com.microsoft.sqlserver.jdbc.SQLServerDriver Backend database driver class used in the deployment of migration Helm chart. Database info configured in value.yaml file. BACKEND_DB_TYPE SQL-Server Backend database database type usedWarning title Important If you are using Jenkins on Windows OS, and have created an agent on Linux OS, you need to do the followings in the export pipeline file.
1. Uncomment the following code snippet.
Code Block language css theme Midnight /* agent { label 'LinuxAgent' } */
Where,
LinuxAgent is the name of the agent that you have created.
2. Comment the following lines of code.
Code Block language css theme Midnight agent any
- Click Save.
On the screen that follows, click Build Now.
As you build the pipeline for the very first time, all the parameters get initialized.Refresh the page.
The Build Now option now changes to Build with Parameters.Click Build with Parameters.
You will see all the parameters inherited from the import pipeline file.
Enter the parameter values as per your requirement.
Info It is mandatory to provide a valid value for all the Jenkins parameters. - Log in to Jenkins.
- Click Build with Parameters.
- Change the values for the following two parameters, and keep the rest of the values as is.
- Enter rollback as the value for the OPERATION parameter.
Enter the path of the rollback ZIP in GitHub as the value for GIT_ROLLBACK_ZIP_PATH parameter. This path will be used for pulling the rollback ZIP from GitHub.
The objects are imported to the target environment.
The rollback ZIP is created and pushed to the GitHub repository.
All the log statements generated by the migration utility during the import process are tagged with a unique identifier. This allows you to search for this unique identifier in the centralized logging system, and fetch all the associated logs.
Enter the path of the rollback ZIP in PVC as the value for SOURCE_ZIP_PATH parameter.
The format of rollback ZIP file name is Recovery_<EXPORT_ZIP_NAME>.zip.Click
Expand | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Parameters used in rollback operation | Nexus NEXUS_CREDENTIALS_ID | nexus_credentialsId | Store Github access credentials in Jenkins. Select credential type "Username and password" and create global credentials in Jenkins. Example: user: valuelabs Pass: password Note: Use credentials binding plugin to store credentials in Jenkins. Select credential type "Username and password" and save it. A Credentials ID can be added or configures in Jenkins. Using credential ID, the credentials can be accessible globally in Jenkins. NEXUS_HELM_REPO_URL | https://nexus.adeptia.com:8443/repository/adeptia-internal-helm | Nexus Helm chart repository URL | CHART_NAME | migration | Helm chart name. | GitHub | GIT_CREDENTIALS_ID | gitCredential_id | Store GitHub access credentials in Jenkins. Select credential type "Username and password" and create global credentials in Jenkins. GIT_REPO_URL | https://github.com/prathi-adeptia/migration-defination.git | GitHub repository URL. | GIT_BRANCH | main | GitHub branch name. | GIT_SOURCE_ZIP_PATH | test/import.xml | Location to pull the import.xml file from GitHub repository. | GIT_MIGRATION_XML_FILE_PATH | test/Rolllback_SA_PF.zip | Location to pull rollback zip to GitHub repository. | Migration | OPERATION | rollback | Specify the type of operation for which deployment will be performed. eg. export, import, rollback | SOURCE_ZIP_PATH | /shared/migrationtest1/Rolllback_SA_PF.zip | Rollback zip path to upload file on shared PVC (K8 cluster) | MIGRATION_XML_FILE_PATH | /shared/migrationtest1/import.xml | Import.xml path to upload file on shared PVC (K8 cluster) | LOG_IDENTIFIER | Test_Identifier_Tag | Log identifier to capture logs from MS environment. | K8 Cluster | K8_CREDENTIALS_ID | k8credentials | Store K8 cluster access credentials in Jenkins. Select credential type "Secret file" and create global credentials in Jenkins. Example: user: valuelabs Pass: Secret file (Here secret file is k8 .config file) SERVER_URL | https://valuelabs-dev-dns-2ce021bb.hcp.eastus.azmk8s.io:443 | Kubernetes cluster URL | CLUSTER_CONTEXT | test-dev | Kubernetes cluster context | NAMESPACE | namespace | Kubernetes cluster namespace for deployment. | Database | BACKEND_DB_URL | jdbc:sqlserver://adeptia-adeptiaconnectdb.database.windows.net:1433;database=valuelabs-backend-8 | Backend database URL used in the deployment of migration Helm chart. Database info configured in value.yaml file. | DATABASE_CREDENTIALS_ID | databasecredentialid | Store database access credentials in Jenkins. Select credential type "Username and password" and create global credentials in Jenkins. | BACKEND_DB_DRIVER_CLASS | com.microsoft.sqlserver.jdbc.SQLServerDriver | Backend database driver class used in the deployment of migration Helm chart. Database info configured in value.yaml file. | BACKEND_DB_TYPE | SQL-Server | Backend database database type used in the deployment of migration Helm chart. Database info configured in value.yaml file. | |
Click Build to trigger the pipeline.
Info |
---|
Rolling back the import operation
If you decide upon rolling back the import operation, follow the instructions given below.
.yaml file. DATABASE_CREDENTIALS_ID <credential ID generated by Jenkins> Credential ID for database in Jenkins.
Refer to the prerequisites for more details.
BACKEND_DB_DRIVER_CLASS com.microsoft.sqlserver.jdbc.SQLServerDriver Backend database driver class used in the deployment of migration Helm chart. Database info configured in values.yaml file. BACKEND_DB_TYPE SQL-Server Backend database type used in the deployment of migration Helm chart. Database info configured in values.yaml file. Click Build to trigger the pipeline.
Info The objects are imported to the target environment.
The rollback ZIP is created and pushed to the GitHub repository.
All the log statements generated by the migration utility during the import process are tagged with a unique identifier. This allows you to search for this unique identifier in the centralized logging system, and fetch all the associated logs.
Rolling back the import operation
If you decide upon rolling back the import operation, use the rollback ZIP created during the import operation.
Here are the steps to perform the rollback operation.
- Log in to Jenkins.
Click Build with Parameters.
Enter the values for the following parameters.
Info It is mandatory to provide a valid value for all the Jenkins parameters, except for GIT_ROLLBACK_ZIP_PATH and ROLLBACK_ZIP_PATH. Expand title Click here to expand the list of rollback parameters Helm Chart
HELM_REPO_URL https://adeptia.github.io/adeptia-connect-migration/charts Migration Helm chart repository URL. CHART_NAME migration Name of the Migration Helm chart. GitHub GIT_CREDENTIALS_ID <credential ID generated by Jenkins>
Credential ID for GitHub in Jenkins.
Refer to the prerequisites for more details.
GIT_REPO_URL https://github.com/adeptia/migration-defination.git URL of the GitHub repository. GIT_BRANCH main GitHub branch name. GIT_SOURCE_ZIP_PATH test/Rolllback_SA_PF.zip Location of the rollback ZIP in GitHub repository.
GIT_MIGRATION_XML_FILE_PATH test/import.xml Location of the import.xml file in GitHub repository. Migration OPERATION rollback The type of operation for which deployment will be performed. SOURCE_ZIP_PATH /shared/migrationtest1/Rolllback_SA_PF.zip Path of the rollback ZIP in the PVC.
Info The rollback ZIP is created at the location where your Exported ZIP is located.
The format of rollback ZIP file name is Recovery_<EXPORT_ZIP_NAME>.zip.MIGRATION_XML_FILE_PATH /shared/migrationtest1/import.xml Path of the import.xml file in the PVC. LOG_IDENTIFIER Test_Identifier_Tag Log identifier to capture logs from MS environment. Kubernetes Cluster K8_CREDENTIALS_ID <credential ID generated by Jenkins> Credential ID for Kubernetes in Jenkins.
Refer to the prerequisites for more details.
SERVER_URL https://<host name of the Kubernetes cluster>:<Port number> URL of the Kubernetes cluster. CLUSTER_CONTEXT test-dev Kubernetes cluster context. NAMESPACE namespace Kubernetes cluster namespace for deployment. Database BACKEND_DB_URL jdbc:sqlserver://<DB Hostname>:<Port number>;database=<Backend Database Name> Backend database URL used in the deployment of migration Helm chart. Database info configured in value.yaml file. DATABASE_CREDENTIALS_ID <credential ID generated by Jenkins> Credential ID for database in Jenkins.
Refer to the prerequisites for more details.
BACKEND_DB_DRIVER_CLASS com.microsoft.sqlserver.jdbc.SQLServerDriver Backend database driver class used in the deployment of migration Helm chart. Database info configured in values.yaml file. BACKEND_DB_TYPE SQL-Server Backend database type used in the deployment of migration Helm chart. Database info configured in values.yaml file. Click Build to roll back the import operation.
Info RetainXmlLocation, OverRideuser, and OverrideModifiedByuser parameters are not applicable in case of rollback.