{"id":13194,"date":"2022-07-25T14:40:04","date_gmt":"2022-07-25T14:40:04","guid":{"rendered":"https:\/\/blog.cloudthat.com\/?p=13194"},"modified":"2024-06-25T10:56:47","modified_gmt":"2024-06-25T10:56:47","slug":"integrate-github-monorepo-aws-code-pipeline","status":"publish","type":"blog","link":"https:\/\/www.cloudthat.com\/resources\/blog\/improving-developer-productivity-github-monorepo-integration-with-aws-code-pipeline-guide","title":{"rendered":"Improving Developer Productivity: GitHub Monorepo Integration with AWS Code Pipeline Guide"},"content":{"rendered":"<table style=\"height: 292px;\" border=\"0\" width=\"338\">\n<tbody>\n<tr>\n<td>\n<h2><span style=\"color: #000080;\"><strong>TABLE OF CONTENT<\/strong><\/span><\/h2>\n<\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#overview\">1. Overview of CodePipeline<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#topfeatures\">2. Top Features of CodePipeline<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#whatismonorepo\">3. What is Monorepo?<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#benefits\">4. Benefits of using Monorepo<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#solutionoverview\">5. Solution Overview<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#architecturediagram\">6. Sample Architecture Diagram<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#lambdafunction\">7. Creating the Lambda Function<br \/>\n<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#githubwebhook\">8. Creating the GitHub Webhook<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#conclusion\">9. Conclusion<\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#aboutcloudthat\">10. About CloudThat <\/a><\/td>\n<\/tr>\n<tr>\n<td><a style=\"margin-left: 20px;\" href=\"#faqs\">11. FAQs<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<h2 id=\"overview\"><strong>Overview of Code Pipeline<\/strong><\/h2>\n<p><span style=\"color: #000000;\">AWS Code Pipeline is an AWS service that helps to deploy the software automatically. It will allow the developer to deliver the code for new updates quickly. It is a continuous delivery service. CodePipeline will automatically build, test, and launch an application every time the code changes.<\/span><\/p>\n<p><span style=\"color: #000000;\">It will reduce the development time, which contains only a few manual steps. With CodePipeline, we can easily configure the software release at different stages.<\/span><\/p>\n<h2 id=\"topfeatures\"><strong>Top Features of CodePipeline<\/strong><\/h2>\n<ul>\n<li><span style=\"color: #000000;\">Access Control: We can manage the user activities and control the user who can make changes to the code by using AWS IAM.<\/span><\/li>\n<li><span style=\"color: #000000;\">Receive Notification: We can integrate Amazon SNS notifications to receive the notifications. Every notification consists of status messages of the events.<\/span><\/li>\n<li><span style=\"color: #000000;\">Fast Delivery: It will allow to rapidly release the new features to the users as and when there is a new release.<\/span><\/li>\n<li><span style=\"color: #000000;\">Easy Integration: We can use pre-built plugins or custom plugins in steps of your release process.<\/span><\/li>\n<li><span style=\"color: #000000;\">Quick Start: It is a service that will continuously manage to deliver the service and connect to the current system and tools.<\/span><\/li>\n<\/ul>\n<h2 id=\"whatismonorepo\"><strong>What is Monorepo?<\/strong><\/h2>\n<p><span style=\"color: #000000;\">Many organizations make use of GitHub as a source code repository. Multiple applications and services store the source code of numerous projects in a single repository, and it is called monorepo.<\/span><\/p>\n<p><span style=\"color: #000000;\">Monorepo is a centralized place where the codebase lives in the same repository. Everyone in the team uses the latest version of code for the projects. It is a single repository that stores the codes of multiple applications and libraries.<\/span><\/p>\n<h2 id=\"benefits\"><strong>Benefits of using Monorepo:<\/strong><\/h2>\n<ol>\n<li><span style=\"color: #000000;\">Shared Code: Code is stored in a single repository, and everyone in the team can share the latest version of the code. This enables to share of the logic between the frontend and backend.<\/span><\/li>\n<li><span style=\"color: #000000;\">Ease of maintenance: When the code is updated in the library, the update is made to all the applications which use the shared library.<\/span><\/li>\n<li><span style=\"color: #000000;\">Atomic changes: With monorepo, there is no need to coordinate with multiple teams or projects if there is any change in the code.<\/span><\/li>\n<li><span style=\"color: #000000;\">Code reuse: Different projects can use the common code stored in the repository.<\/span><\/li>\n<li><span style=\"color: #000000;\">Faster code review: We can quickly track and review the code changes when the code is stored in a single repository.<\/span><\/li>\n<\/ol>\n<h2 id=\"solutionoverview\"><strong>Solution Overview<\/strong><\/h2>\n<p><span style=\"color: #000000;\">The release pipeline is invoked by default when there is a change in the code repository. While using GitHub as the pipeline source the pipeline, to detect the changes in the remote branch and start the pipeline, the CodePipeline will make use of a webhook. When we make changes in any of the folders in the repository of the monorepo with GitHub, the CodePipeline gets an event at the repository level.<\/span><\/p>\n<p><a href=\"https:\/\/d1f7lmxeo98xps.cloudfront.net\/resources\/wp-content\/uploads\/2022\/11\/monorepo1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13195\" src=\"https:\/\/d1f7lmxeo98xps.cloudfront.net\/resources\/wp-content\/uploads\/2022\/11\/monorepo1.png\" alt=\"Monorepo\" width=\"285\" height=\"228\" \/><\/a><\/p>\n<p><span style=\"color: #000000;\">The above diagram describes how GitHub events will invoke a monorepo service-specific pipeline by evaluating the event triggered using Lambda.<\/span><\/p>\n<ol>\n<li><span style=\"color: #000000;\">Add customizations to start pipeline based on external factors: Whether a pipeline should be triggered or not can be customized using custom code. We can create custom logic to trigger the pipelines.<\/span><\/li>\n<li><span style=\"color: #000000;\">Have multiple pipelines with a single source: By making groups, we can make changes to the selected pipelines, when multiple pipelines use single repo.<\/span><\/li>\n<li><span style=\"color: #000000;\">Avoid reacting to unimportant files: When changing the file doesn\u2019t affect the application, we can avoid triggering the pipeline.<\/span><\/li>\n<\/ol>\n<h2 id=\"architecturediagram\"><strong>Sample Architecture Diagram<\/strong><\/h2>\n<p><a href=\"https:\/\/d1f7lmxeo98xps.cloudfront.net\/resources\/wp-content\/uploads\/2022\/11\/monorepo2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13196\" src=\"https:\/\/d1f7lmxeo98xps.cloudfront.net\/resources\/wp-content\/uploads\/2022\/11\/monorepo2.png\" alt=\"Monorepo\" width=\"480\" height=\"285\" \/><\/a><\/p>\n<p><span style=\"color: #000000;\">The above figure describes the following steps:<\/span><\/p>\n<ol>\n<li><span style=\"color: #000000;\">Changes to the source code are completed and stored in the GitHub repo.<\/span><\/li>\n<li><span style=\"color: #000000;\">The push event will be triggered by the Webhook.<\/span><\/li>\n<li><span style=\"color: #000000;\">API Gateway will authenticate the GitHub webhook push event and then invoke the lambda function<\/span><\/li>\n<li><span style=\"color: #000000;\">Post Lambda function is invoked, it will check for the configuration files that are stored in the S3 bucket.<\/span><\/li>\n<li><span style=\"color: #000000;\">Lambda function will start the CodePipeline.<\/span><\/li>\n<\/ol>\n<p><span style=\"color: #000000;\">The above solution describes the following parts:<\/span><\/p>\n<ul>\n<li><span style=\"color: #000000;\"><strong><a href=\"https:\/\/blog.cloudthat.com\/creation-of-api-iot-application-aws-api-gateway\/?utm_source=blog-website&amp;utm-medium=text-link&amp;utm_campaign=creation-of-api-iot-application-aws-api-gateway\" target=\"_blank\" rel=\"noopener\">Amazon API Gateway<\/a>:<\/strong> It will receive the authentication GitHub webhook push event from the GitHub Repo. It will also evaluate the push event incoming from GitHub and start the pipeline<\/span><\/li>\n<li><span style=\"color: #000000;\"><strong>Amazon S3:<\/strong> S3 bucket will store the specific CodePipeline configuration files<\/span><\/li>\n<li><span style=\"color: #000000;\"><strong><a href=\"https:\/\/blog.cloudthat.com\/awscodebuild\/?utm_source=blog-website&amp;utm-medium=text-link&amp;utm_campaign=awscodebuild\" target=\"_blank\" rel=\"noopener\">AWS CodeBuild<\/a>:<\/strong> It contains the build stage of the pipeline<\/span><\/li>\n<\/ul>\n<p><span style=\"color: #000000;\">The CodePipeline will automatically trigger the pipeline to release the latest version of the source code. By choosing the release change we can manually run the latest version of the code.<\/span><\/p>\n<h2 id=\"lambdafunction\"><strong>Creating the Lambda Function<\/strong><\/h2>\n<p><span style=\"color: #000000;\">The Lambda function is responsible for evaluating and authenticating the events. The function can be resolved through GitHub event payloads by determining which files are changed, deleted, added and appropriate actions are performed:<\/span><\/p>\n<ul>\n<li><span style=\"color: #000000;\">The single pipeline is started depending on the folder changes in GitHub.<\/span><\/li>\n<li><span style=\"color: #000000;\">Multiple Pipelines are started.<\/span><\/li>\n<li><span style=\"color: #000000;\">If non-relevant files are changed, ignore the change.<\/span><\/li>\n<\/ul>\n<p><span style=\"color: #000000;\">S3 stores the project configuration details, Lambda can read the configuration file and decides what actions to be taken when the particular file is matched from the GitHub event.<\/span><\/p>\n<p><a href=\"https:\/\/d1f7lmxeo98xps.cloudfront.net\/resources\/wp-content\/uploads\/2022\/11\/MicrosoftTeams-image-91.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13643\" src=\"https:\/\/d1f7lmxeo98xps.cloudfront.net\/resources\/wp-content\/uploads\/2022\/11\/MicrosoftTeams-image-91.png\" alt=\"Monorepo\" width=\"338\" height=\"220\" \/><\/a><\/p>\n<h2 id=\"githubwebhook\"><strong>Creating the GitHub Webhook<\/strong><\/h2>\n<p><span style=\"color: #000000;\">Webhooks allow external services to be notified of certain events provided by GitHub. Webhooks are used to create to push events. POST requests are generated to the URL specified for any files committed and pushed to the repository.<\/span><\/p>\n<p><a href=\"https:\/\/d1f7lmxeo98xps.cloudfront.net\/resources\/wp-content\/uploads\/2022\/11\/monorepo4.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13198\" src=\"https:\/\/d1f7lmxeo98xps.cloudfront.net\/resources\/wp-content\/uploads\/2022\/11\/monorepo4.png\" alt=\"Monorepo\" width=\"505\" height=\"321\" \/><\/a><\/p>\n<h2 id=\"conclusion\"><strong>Conclusion<\/strong><\/h2>\n<p><span style=\"color: #000000;\">In the above example, the GitHub source code repository is monitored by two pipelines. Based on the GitHub events, the lambda function will decide which pipeline to run. The lambda function has the logic to ignore the unimportant file. Using S3, API Gateway, and Lambda combination servers as logic to invoke pipeline.<\/span><\/p>\n<h2 id=\"aboutcloudthat\"><strong>About CloudThat<\/strong><\/h2>\n<p><span style=\"color: #000000;\"><a href=\"https:\/\/www.cloudthat.com\/\"><strong>CloudThat\u00a0<\/strong><\/a>is the official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner and Microsoft gold partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best in industry cloud computing practices and expertise. We are on a mission to build\u00a0a robust\u00a0cloud computing ecosystem by disseminating\u00a0knowledge on technological intricacies within the cloud space.\u00a0Our blogs, webinars,\u00a0case studies, and white papers\u00a0enable all the stakeholders in the cloud computing sphere.<\/span><\/p>\n<p><span style=\"color: #000000;\">Drop a query if you have any questions regarding Github Integration, AWS CodePipeline, and DevOps best practices and I will get back to you quickly. To get started, go through\u00a0our<strong>\u00a0<\/strong><a href=\"https:\/\/www.cloudthat.com\/expert-advisory\/?utm_source=blog-website&amp;utm-medium=text-link&amp;utm_campaign=expert-advisory\"><strong>Expert Advisory<\/strong>\u00a0<\/a>page\u00a0and\u00a0<a href=\"https:\/\/www.cloudthat.com\/managed-services-packages\/\"><strong>Managed Services Package<\/strong><\/a>\u00a0that is<strong>\u00a0<a href=\"https:\/\/cloudthat.com\/?utm_source=blog-website&amp;utm-medium=text-link&amp;utm_campaign=cloudthat.com\/\">CloudThat<\/a>\u2019s<\/strong>\u00a0offerings.<\/span><\/p>\n<h2 id=\"faqs\"><strong>FAQs<\/strong><\/h2>\n<ol>\n<li>\n<h4><span style=\"text-decoration: underline; color: #000000;\"><strong>Can multiple projects be put in a single repository?<\/strong><\/span><\/h4>\n<\/li>\n<\/ol>\n<p><span style=\"color: #000000;\">Yes. Multiple projects can be put in a single repository, but they would have to be on different branches.<\/span><\/p>\n<ol start=\"2\">\n<li>\n<h4><span style=\"text-decoration: underline; color: #000000;\"><strong>When to use monorepo and when multi-repo?<\/strong><\/span><\/h4>\n<\/li>\n<\/ol>\n<p><span style=\"color: #000000;\">Monorepos can be used when the entire team can view the changes done to the code by a single person. Multi-Repos creates separate repos for every team.<\/span><\/p>\n","protected":false},"author":312,"featured_media":13636,"parent":0,"comment_status":"open","ping_status":"open","template":"","blog_category":[3606,3607,3624],"user_email":"sindhuh@cloudthat.com","published_by":"324","primary-authors":"","secondary-authors":"","acf":[],"_links":{"self":[{"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/blog\/13194"}],"collection":[{"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/blog"}],"about":[{"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/types\/blog"}],"author":[{"embeddable":true,"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/users\/312"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/comments?post=13194"}],"version-history":[{"count":3,"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/blog\/13194\/revisions"}],"predecessor-version":[{"id":45887,"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/blog\/13194\/revisions\/45887"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/"}],"wp:attachment":[{"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/media?parent=13194"}],"wp:term":[{"taxonomy":"blog_category","embeddable":true,"href":"https:\/\/www.cloudthat.com\/resources\/wp-json\/wp\/v2\/blog_category?post=13194"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}