In this article, we will explain how to create a pull request into Merge Pipelines into Master Branch in Azure Data Factory.
Pre-requisites to create feature branch to Merge pipelines into Master:
- Master branch is our collaboration branch for each ADF instances.
- Developers should create their own feature(working/dev) branches for each project from the Master branch in their respective dev ADF instance.
- Developers should only SAVE their changes in their feature branch
- Once developers are ready for code promotion, they should open a PULL REQUEST. Before Open a pull request, developer should update feature branch with master branch and resolve any conflict.
- A pull request serves two proposes: i) establishes code review process and ii) ensures code merge tasks follow a standard process.
The following pointers must be completed before the pull request raise in ADF:
- Clean the respective branch by removing unwanted pipelines (Test, copy) and keep only those which are needed.
- Make sure all the Datasets, pipelines, etc., are present as per naming conventions provided by user.
- Azure Blob Storage should point to the respective Azure ADF Environments that we are creating.
- Ensure, NO HARDCODED VALUES are present for any datasets or variables and parameters in ADF Pipeline. HARD CODED values that are being used in ADF Pipelines, is not the best practice and should be parametrized in ADF Pipelines.
- Dataset should be moved to their respective folders and should not be present at the root level.
- Email Notifications should be set to the respective status. For success notification it should be “success” and for failure notification it should be “failed”.
- Environmental variables should be verified, and it should have correct values with respect to the environment entries.
- Hard coded values should be removed for success and failure stored procedure (Transformation pipelines).
- Delete all the Datasets that are created by our team during development phase and are not used in any of the pipelines. We do not have to delete the Datasets created by other teams or other Projects.
- Create a pull request by selecting your branch in DEV ADF —> create new pull request –> select adf account —-> your branch—> into –> adf_dev –-> master.
- For the datasets that do not belong to our team, make sure it is not edited by comparing it with master branch. It should not show any difference during comparison. If there are no changes seen, leave them as is.
- To check, who created the datasets : Go to Branches Tab on the left –> Filter your branch –> Expand the “datasets” on the left –> pick the dataset you want to check and check history. For all the datasets that are created by our team and are not used should be deleted. For all the datasets that are not created by us and has no changes seen when compared should be left as is.
- The support pipelines should not be edited and should not show any difference when compared to master branch if they are present in your pipelines section after the pull request.
- To revert back the changes for support pipelines and datasets which does not belong to you, and which are in conflict can be done by downloading the code by : go to Branches –> search for master branch –> click on datasets or pipelines folders and look for DS or pipeline you want the code of –> click on 3 dots of the particular DS or pipeline and click on download –> search for your branch –> click on 3 dots of DS folder or pipeline folder and upload the file. This exercise will get all the code from master as it is in your branch for which values might have been changed.
- All the newly created pipelines should be ignored as no comparison will be available due to newly creation of these pipelines.
- Triggers: Triggers will be in deactivated state in DEV ADF. If in the pull request, other triggers apart from your triggers are seen then there will be a difference in code as compared with master. In this scenario, download those triggers from master and upload in your DEV branch.
- All the conflicts are to be solved that are shown for a pull request.
- Created through Azure DevOps
- Mechanism for developers to notify team members.
- What the changes are?
- Who requested the changes?
- When should the changes be implemented?
- Maintains documents and history of changes
- SQL scripts and Test Evidence Documents
- Dedicated forum to discuss the proposed change.
- Teammates can review the changes.
- Share feedback
Pull request sample:
In your pull request, indicate which branch that needs to be merged to the master branch.
Provide the appropriate title for your pull request
Add additional information in the description.
After assigned reviewers (like managers,architeture level) click on create option.
Once PR is approved by reviewers our pipelines are merged into master.